There are a lot of snippets in the examples/example.rb (in a hardly readable format). Some of the examples are in separate files (I guess it was easier to create a new file than follow the pattern in the examples.rb). The comments around the snippets not always tell you what it really does, etc. We need to clean up these files to be more helpful.

There are ongoig threads about this topic:

• #32 Pointing out that some of the files are a bit outdated
• #41 Solving the previouos issue with some suggestions
• https://github.com/caxlsx/caxlsx/pull/34#issuecomment-567679024 Describing how we usually want to use the examples during gem development

Based on the above we want to achieve these goals:

• [ ] Unified code samples (easy to understand and spot the difference when they look the same)
• [ ] Descriptive examples (descriptions and / or images how the output looks like)
• [ ] Discoverable examples (categories or easily searchable lists)
• [ ] Separate files (hard to find anything useful in the big example.rb)
• [ ] Runnable examples (for manual testing)

FIxed by #47.

I think this is a vast improvement for caxlx’s documentation, so thanks again @kiskoza for your amazing work!

I wonder if we should somehow integrate this into the test suite and/or ci checks? Like designating the current example documents as “known good” state, and having each test run regenerate and compare the documents to make sure there are no inadvertent changes?

This is my next idea to how to solve #46 : Bring the content of the wiki to the examples folder.

• [x] Unified code samples: Easy to check if new files has a short, informative code snippet
• [x] Descriptive examples: Easy to check if new files has description and image about the output
• [x] Discoverable examples: In examples/README.md we could sort the examples by categories, etc
• [x] Separate files: All examples will be in a separate file
• [x] Runnable examples: I added a example_runner.rb* that can extract the snippets from the markdown files and run all of them, or specific ones

* I used this name because we already have an example.rb that I didn't want to delete right now, but we could reuse that name if we decide to migrate all examples to this format

Unfortunately I don't have time to investigate this myself, but if someone comes up with a fix I'd be more than happy to merge it.

Unfortunately I don't have time to investigate this myself, but if someone comes up with a fix I'd be more than happy to merge it.

Good question, hm.

I don't have any experience with these things: Are there any other areas where forked and “normal” repos behave differently? Would it be possible to convert the repo back to a fork, if this should ever be necessary?

Please add an additional test case for serialize(@fname, true, zip: "zip") as mentioned above.

I think this fails if someone calls the method like this: serialize("foo.xlsx", true, zip: "zip"). Eventhough I suspect this case is highly unlikely, it should nevertheless be handled correctly to make this change really backwards-compatible.

Does it make sense to retain such (escaped) characters in sheet names? In cells for example they are removed, see Cell#clean_value which calls Axlsx.sanitize which removes control characters.

There have been quite a few changes / fixes since the last release, so let's go for it!

(Having that said, maybe we should wait for #57 and include it also?)

@rylwin Would you like to become a member of this project? We’d love to have more people on board making high-quality contributions (like for example this PR)! 😃

Merged! Thanks for bringing this up and implementing it!

Add a :zip_command option to Axlsx::Package#serialize that allows the user to specify an alternate command to use to perform the zip operation on the XLSX file contents.

The default zip operation is provided by RubyZip. On large documents users may experience faster zip times using a zip binary.

Resolves #55

We're starting to generate some pretty large Excel files with caxlsx and, as our files get larger, so do the resources consumed to generate the file. We're working on ways to decrease the time it takes to render Excel files, part of which is due to slowness in our own app (slow queries, etc) and part of which is due to generating the Excel file.

One area we've found for potentially decreasing the time it takes to generate the Excel file is by using the system zip tool instead of rubyzip. We've hacked together a solution that seems to work for us and yield some pretty great improvements.

We have a gist to demonstrate the performance of the "built-in" rubyzip that's currently used by caxlsx vs. the "system-zip" binary: https://gist.github.com/rylwin/013eaa9768e94b75572d34e1695ac472.

On my personal laptop, the benchmark yields:

$ ruby zip-perf.rb
                  user     system      total        real
built-in:     0.503427   0.258035   0.761462 (  0.597083)
system-zip:   0.169251   0.000000   0.202789 (  0.201150)

And on a Heroku instance:

$ ruby zip-perf.rb
                  user     system      total        real
built-in:     2.760000   2.176000   4.936000 (  4.260702)
system-zip:   0.424000   0.000000   0.452000 (  0.483958)

In both cases, the speed up is quite drastic.

If there's interest in incorporating the use of an alternate zip library, I'd be happy to work on a PR. A couple thoughts:

• Shelling out to the system zip binary adds a dependency on zip, which may not always be there. Perhaps via config the user could select how they want to render the Excel file, or we could check for the presence of zip
• I'm on a linux box (I would assume Mac would work fine, not sure about Windows, more exploration would be necessary)
• I tried using an old gem I found called zipruby that provides libzip bindings, but it's last commit is ~7 years ago and when I load it, it conflicts with rubyzip, so while I'd prefer to use something like zipruby to avoid shelling out to zip, that would be a bigger lift, but perhaps worth it.

So that's where we are as of now. Curious to hear the community's thoughts and whether any of this is of interest.

P.S. Thanks for maintaining this great gem!

I think that's a good idea, but let's do this in a separate PR.

We should then cut a release containing both changes, so that users will have to update their code only once.

Ah, one more thing: Please add an entry to the Changelog, too.

@rylwin Marvellous! We’re almost there: Please squash the existing commits into a single one, and add a second one that removes the rescue-parts from the other tests (see above). Then it's merge time! 🚀

Thanks a lot for researching this! I think we should delete these statements from the other tests, too. It's not directly in the scope of this PR, but let's include it here for the sake of simplicity. Make this a separate commit, the text from your comment above can be used more or less 1:1 as commit message, I think.

My bad, sorry! The syntax is indeed correct.

Altogether this is wonderfully written code 😍

Is this necessary to make the test run on CI?

Do we need to shellescape @dir also?

Why not use FileUtils.mkdir_p and File.dirname? (This would also solve any issues regarding escaping the filename.)

Looks like leftovers from a different implementation?

That’s perfect. (We should also change the confirm_valid argument to a keyword argument at some point, but this needs to go through a deprecation cycle first.)

Then let's go for zip_command! 🚀

👍

I’d say let’s keep this issue / PR focused on the external zip option for now.

Reconsidering the name and usage of the option: I just realized that we can probably always require a string – either the full path to a specific zip executable, or simply "zip" to use the first zip executable found in the search path. (I imagine we could simply pass the command name to #system and let the shell do all lookups and path handling.)

Here’s how it would look like for the case when just using "zip":

serialize(external_zip: "zip")
serialize(zip_provider: "zip")
serialize(zip_command: "zip")
serialize(zip_binary: "zip")
serialize(zip_executable: "zip")
serialize(zip: "zip")

I think I’d prefer zip_command: "zip", eventhough this somehow implies that arguments can also be specified. But why not? Maybe it would even make sense performancewise to use zip_command: "zip -1" (“compress faster”)? Or is this too confusing or hard to implement?

Regarding #to_stream I think you already pointed out the problem that it could be surprising if (temporary) files are created when using this method. But I’m not sure: Maybe Rubyzip creates temporary files anyway? In this case (which probably should be documented) it wouldn’t be very much of a difference to use an external zip program. Would you like to investigate this further?

Or maybe we should add another way of serializing a package to better fit these use-cases. Maybe something like

package.serialize do |temporary_package_file|
  # do something with the file
end # temporary package file cleaned up automatically

Good point about the problem with #to_stream supporting this option. Out of curiosity: Wouldn’t it be possible to implement your usecase like this (untested)?

Tempfile.open do |tempfile|
  package.serialize(tempfile.path, external_zip: true)
  send_data tempfile.rewind.read # or even `send_file tempfile.path`
end

I’m not against a config option per se, but I’m a bit hesitant because so far Axlsx has no configuration at all – not sure if it's worth introducing one for a single option only.

I also encounter this bug. (And forcing a sync by restarting the desktop client fixes also fixes it for me.)

One additional note: Some macOS files (“packages”) are actually folders under the hood (Pixelmator documents for example). These files are affected by this bug, too – which can be quite surprising at the first moment, because in Finder they look like normal files, and they have the green checkmark showing that they’ve been synced successfully. Only after using “Show package contents” in Finder (or looking at the file in the webinterface) one notices that the actual contents have not been synced.

I think that's a great idea! Maybe this could be an option in Axlsx::Package#serialize? Here's a spontaneous idea:

package.serialize("test.xlsx", external_zip: true) # uses zip executable in path
package.serialize("test.xlsx", external_zip: "/my/special/zip") # uses specified executable

Thoughts?

Prior to this change, strings like "1e12345" would be interpreted as float values, regardless of the actual value of the exponent (which easily could be out of range for Ruby).

In case the exponent was greater than Float::MAX_10_EXP (usually 308), this would result in a cell of type :float containing the literal string "Infinity". Excel can not parse such cells and therefore gives a “corrupt data” error.

In case the exponent was less than Float::MIN_10_EXP (usually -307) the cell would contain 0.0. This does not result in Excel throwing an error, but probably isn't the expected result either.

Note that this problem is quite likely to happen when creating a worksheet with hexadecimal strings, because e.g. "1234e567" is a perfectly valid hex value.

The additional range check of the exponent introduces a slight performance overhead, so I decided to split the code path: I presume parsing floats with exponents < 100 (or no exponents at all) is way more common, so this code path behaves exactly like before. Only in the case of a 3 digit exponent the additional range check is introduced.

Here is a quick benchmark:

require "benchmark"
require "axlsx"

row = Axlsx::Workbook.new.add_worksheet.add_row
Benchmark.bm(11) do |x|
  x.report("no exp     "){ 1000000.times{ Axlsx::Cell.new(row, "123.23") } }
  x.report("valid exp  "){ 1000000.times{ Axlsx::Cell.new(row, "123.23e45") } }
  x.report("invalid exp"){ 1000000.times{ Axlsx::Cell.new(row, "123.23e456") } }
end

Before this change:

                  user     system      total        real
no exp        2.670989   0.015253   2.686242 (  2.713402)
valid exp     2.838740   0.014406   2.853146 (  2.871085)
invalid exp   2.743406   0.013243   2.756649 (  2.777175)

After this change:

                  user     system      total        real
no exp        2.587773   0.008056   2.595829 (  2.602749)
valid exp     2.744830   0.006783   2.751613 (  2.759950)
invalid exp   4.219549   0.013698   4.233247 (  4.248091)

Prior to this change, strings like "1e12345" would be interpreted as float values, regardless of the actual value of the exponent (which easily could be out of range for Ruby).

In case the exponent was greater than Float::MAX_10_EXP (usually 308), this would result in a cell of type :float containing the literal string "Infinity". Excel can not parse such cells and therefore gives a “corrupt data” error.

In case the exponent was less than Float::MIN_10_EXP (usually -307) the cell would contain 0.0. This does not result in Excel throwing an error, but probably isn't the expected result either.

Note that this problem is quite likely to happen when creating a worksheet with hexadecimal strings, because e.g. "1234e567" is a perfectly valid hex value.

The additional range check of the exponent introduces a slight performance overhead, so I decided to split the code path: I presume parsing floats with exponents < 100 (or no exponents at all) is way more common, so this code path behaves exactly like before. Only in the case of a 3 digit exponent the additional range check is introduced.

I think there have been two or three small fixes since the last release, but they are already merged and mentioned in the Changelog (section “Unreleased”). Nothing else on my radar, currently. Let's do a release! 🚀

Maybe changing line 167 like this fixes the problem?

zip.write IO.read(part[:path], mode: "rb")

Merged, thanks!

Maximum column width limit in MS Excel is 255 characters https://support.microsoft.com/en-us/office/excel-specifications-and-limits-1672b34d-7043-467e-8e27-269d656771c3

If this value is exceeded, MS Excel gets stuck and doesn't allow column resizing.

Ported from https://github.com/randym/axlsx/pull/653

That looks great! This repo isn‘t actively maintained at the moment though, but I’d love to merge this into the community continued version at https://github.com/caxlsx/caxlsx – could you open a PR over there?

Thanks!