Skip to content

Commit 4d0e021

Browse files
committed
Initialize stream writer support
- Add 4 new functions: new_stream_writer, add_table, set_row and flush - Update unit tests and docs for the function - Upgrade the dependencies package version - Using form template for GitHub issues
1 parent c7199a6 commit 4d0e021

File tree

12 files changed

+436
-125
lines changed

12 files changed

+436
-125
lines changed

.github/ISSUE_TEMPLATE/bug_report.md

Lines changed: 0 additions & 50 deletions
This file was deleted.

.github/ISSUE_TEMPLATE/bug_report.yml

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
name: Bug report
2+
description: Create a report to help us improve
3+
body:
4+
- type: markdown
5+
attributes:
6+
value: |
7+
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
8+
9+
- type: textarea
10+
id: description
11+
attributes:
12+
label: Description
13+
description: Briefly describe the problem you are having in a few paragraphs.
14+
validations:
15+
required: true
16+
17+
- type: textarea
18+
id: reproduction-steps
19+
attributes:
20+
label: Steps to reproduce the issue
21+
description: Explain how to cause the issue in the provided reproduction.
22+
placeholder: |
23+
1.
24+
2.
25+
3.
26+
validations:
27+
required: true
28+
29+
- type: textarea
30+
id: received
31+
attributes:
32+
label: Describe the results you received
33+
validations:
34+
required: true
35+
36+
- type: textarea
37+
id: expected
38+
attributes:
39+
label: Describe the results you expected
40+
validations:
41+
required: true
42+
43+
- type: input
44+
id: py-version
45+
attributes:
46+
label: Python version
47+
description: |
48+
Output of `python --version`:
49+
placeholder: e.g. 3.8.10
50+
validations:
51+
required: true
52+
53+
- type: input
54+
id: excelize-version
55+
attributes:
56+
label: Excelize version or commit ID
57+
description: |
58+
Which version of Excelize are you using?
59+
placeholder: e.g. 0.0.1
60+
validations:
61+
required: true
62+
63+
- type: textarea
64+
id: env
65+
attributes:
66+
label: Environment
67+
description: Environment details (OS, Microsoft Excel™ version, physical, etc.)
68+
render: shell
69+
validations:
70+
required: true
71+
72+
- type: checkboxes
73+
id: checkboxes
74+
attributes:
75+
label: Validations
76+
description: Before submitting the issue, please make sure you do the following
77+
options:
78+
- label: Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
79+
required: true
80+
- label: The provided reproduction is a minimal reproducible example of the bug.
81+
required: true

.github/ISSUE_TEMPLATE/feature_request.md

Lines changed: 0 additions & 50 deletions
This file was deleted.
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
name: Feature request
2+
description: Suggest an idea for this project
3+
body:
4+
- type: markdown
5+
attributes:
6+
value: |
7+
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
8+
9+
- type: textarea
10+
id: description
11+
attributes:
12+
label: Description
13+
description: Describe the feature that you would like added
14+
validations:
15+
required: true
16+
17+
- type: textarea
18+
id: additional-context
19+
attributes:
20+
label: Additional context
21+
description: Any other context or screenshots about the feature request here?
22+
23+
- type: checkboxes
24+
id: checkboxes
25+
attributes:
26+
label: Validations
27+
description: Before submitting the issue, please make sure you do the following
28+
options:
29+
- label: Check that there isn't already an issue that requests the same feature to avoid creating a duplicate.
30+
required: true

excelize.py

Lines changed: 139 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -439,10 +439,89 @@ def py_value_to_c_interface(py_value):
439439
return py_value_to_c(interface, types_go._Interface())
440440

441441

442+
class StreamWriter:
443+
sw_index: int
444+
445+
def __init__(self, sw_index: int):
446+
self.sw_index = sw_index
447+
448+
def add_table(self, table: Table) -> Optional[Exception]:
449+
"""
450+
Creates an Excel table for the stream writer using the given cell range
451+
and format set.
452+
453+
Note that the table must be at least two lines including the header. The
454+
header cells must contain strings and must be unique. Currently, only
455+
one table is allowed for a stream writer. The function must be called
456+
after the rows are written but before 'flush'.
457+
458+
Args:
459+
table (Table): The table options
460+
461+
Returns:
462+
Optional[Exception]: Returns None if no error occurred,
463+
otherwise returns an Exception with the message.
464+
465+
Example:
466+
For example, create a table of A1:D5 on Sheet1:
467+
468+
.. code-block:: python
469+
470+
err = sw.add_table(excelize.Table(range="A1:D5"))
471+
"""
472+
lib.StreamAddTable.restype = c_char_p
473+
options = py_value_to_c(table, types_go._Table())
474+
err = lib.StreamAddTable(self.sw_index, byref(options)).decode(ENCODE)
475+
return None if err == "" else Exception(err)
476+
477+
def set_row(
478+
self,
479+
cell: str,
480+
values: List[Union[None, int, str, bool, datetime, date]],
481+
) -> Optional[Exception]:
482+
"""
483+
Writes an array to stream rows by giving starting cell reference and a
484+
pointer to an array of values. Note that you must call the 'flush'
485+
function to end the streaming writing process.
486+
487+
Args:
488+
cell (str): The cell reference
489+
values (List[Union[None, int, str, bool, datetime, date]]): The cell
490+
values
491+
492+
Returns:
493+
Optional[Exception]: Returns None if no error occurred,
494+
otherwise returns an Exception with the message.
495+
"""
496+
lib.StreamSetRow.restype = c_char_p
497+
vals = (types_go._Interface * len(values))()
498+
for i, value in enumerate(values):
499+
vals[i] = py_value_to_c_interface(value)
500+
err = lib.StreamSetRow(
501+
self.sw_index,
502+
cell.encode(ENCODE),
503+
byref(vals),
504+
len(vals),
505+
).decode(ENCODE)
506+
return None if err == "" else Exception(err)
507+
508+
def flush(self) -> Optional[Exception]:
509+
"""
510+
Ending the streaming writing process.
511+
512+
Returns:
513+
Optional[Exception]: Returns None if no error occurred,
514+
otherwise returns an Exception with the message.
515+
"""
516+
lib.StreamFlush.restype = c_char_p
517+
err = lib.StreamFlush(self.sw_index).decode(ENCODE)
518+
return None if err == "" else Exception(err)
519+
520+
442521
class File:
443522
file_index: int
444523

445-
def __init__(self, file_index):
524+
def __init__(self, file_index: int):
446525
self.file_index = file_index
447526

448527
def save(self, *opts: Options) -> Optional[Exception]:
@@ -1660,6 +1739,65 @@ def new_sheet(self, sheet: str) -> Tuple[int, Optional[Exception]]:
16601739
err = res.err.decode(ENCODE)
16611740
return res.val, None if err == "" else Exception(err)
16621741

1742+
def new_stream_writer(
1743+
self, sheet: str
1744+
) -> Tuple[Optional[StreamWriter], Optional[Exception]]:
1745+
"""
1746+
Returns stream writer struct by given worksheet name used for writing
1747+
data on a new existing empty worksheet with large amounts of data. Note
1748+
that after writing data with the stream writer for the worksheet, you
1749+
must call the 'flush' method to end the streaming writing process,
1750+
ensure that the order of row numbers is ascending when set rows, and the
1751+
normal mode functions and stream mode functions can not be work mixed to
1752+
writing data on the worksheets. The stream writer will try to use
1753+
temporary files on disk to reduce the memory usage when in-memory chunks
1754+
data over 16MB, and you can't get cell value at this time.
1755+
1756+
Args:
1757+
sheet (str): The worksheet name
1758+
1759+
Returns:
1760+
Tuple[Optional[StreamWriter], Optional[Exception]]: A tuple
1761+
containing stream writer object if successful, or None and an
1762+
Exception if an error occurred.
1763+
1764+
Example:
1765+
For example, set data for worksheet of size 102400 rows x 50 columns
1766+
with numbers:
1767+
1768+
.. code-block:: python
1769+
1770+
import excelize, random
1771+
1772+
f = excelize.new_file()
1773+
sw, err = f.new_stream_writer("Sheet1")
1774+
if err:
1775+
print(err)
1776+
for r in range(2, 102401):
1777+
row = [random.randrange(640000) for _ in range(1, 51)]
1778+
cell, err = excelize.coordinates_to_cell_name(1, r, False)
1779+
if err:
1780+
print(err)
1781+
err = sw.set_row(cell, row)
1782+
if err:
1783+
print(err)
1784+
err = sw.flush()
1785+
if err:
1786+
print(err)
1787+
err = f.save_as("Book1.xlsx")
1788+
if err:
1789+
print(err)
1790+
err = f.close()
1791+
if err:
1792+
print(err)
1793+
"""
1794+
lib.NewStreamWriter.restype = types_go._IntErrorResult
1795+
res = lib.NewStreamWriter(self.file_index, sheet.encode(ENCODE))
1796+
err = res.err.decode(ENCODE)
1797+
if err == "":
1798+
return StreamWriter(res.val), None
1799+
return None, Exception(err)
1800+
16631801
def new_style(self, style: Style) -> Tuple[int, Optional[Exception]]:
16641802
"""
16651803
Create the style for cells by a given style options, and returns style

go.mod

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ module github.com/xuri/excelize-py
33
go 1.20
44

55
require (
6-
github.com/xuri/excelize/v2 v2.9.1-0.20241229043028-3f6ecffcca89
6+
github.com/xuri/excelize/v2 v2.9.1-0.20250105013731-af422e17009b
77
golang.org/x/image v0.23.0
88
)
99

go.sum

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,8 @@ github.com/tiendc/go-deepcopy v1.2.0 h1:6vCCs+qdLQHzFqY1fcPirsAWOmrLbuccilfp8UzD
1010
github.com/tiendc/go-deepcopy v1.2.0/go.mod h1:toXoeQoUqXOOS/X4sKuiAoSk6elIdqc0pN7MTgOOo2I=
1111
github.com/xuri/efp v0.0.0-20241211021726-c4e992084aa6 h1:8m6DWBG+dlFNbx5ynvrE7NgI+Y7OlZVMVTpayoW+rCc=
1212
github.com/xuri/efp v0.0.0-20241211021726-c4e992084aa6/go.mod h1:ybY/Jr0T0GTCnYjKqmdwxyxn2BQf2RcQIIvex5QldPI=
13-
github.com/xuri/excelize/v2 v2.9.1-0.20241229043028-3f6ecffcca89 h1:KlbACs1A0q8+TJ3keiTh6ir/L0HBFwNe416d1c+dez4=
14-
github.com/xuri/excelize/v2 v2.9.1-0.20241229043028-3f6ecffcca89/go.mod h1:NBRx6e5FHFx4mHLiYG1QBONNvNNSs/wrtzS+h56/A6k=
13+
github.com/xuri/excelize/v2 v2.9.1-0.20250105013731-af422e17009b h1:IgzLFLfCA10Lu5Ac9XgC8wBvPAVqHe2ilgIA8nouiPc=
14+
github.com/xuri/excelize/v2 v2.9.1-0.20250105013731-af422e17009b/go.mod h1:NBRx6e5FHFx4mHLiYG1QBONNvNNSs/wrtzS+h56/A6k=
1515
github.com/xuri/nfp v0.0.0-20240318013403-ab9948c2c4a7 h1:hPVCafDV85blFTabnqKgNhDCkJX25eik94Si9cTER4A=
1616
github.com/xuri/nfp v0.0.0-20240318013403-ab9948c2c4a7/go.mod h1:WwHg+CVyzlv/TX9xqBFXEZAuxOPxn2k1GNHwG41IIUQ=
1717
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=

0 commit comments

Comments
 (0)