Skip to content
\n

On smaller arrays I would do:

\n
from rioxarray import merge\n\narrays = [first_array, second_array]\nmerged_array = merge.merge_arrays(arrays, nodata=0)\n
\n

However... on my big datasets, that doesn't work. I encounter pretty quickly a MemoryError.

\n

For now, the only solution I found is:

\n
    \n
  1. convert the zarr to geotifs files with first_array.to_raster(\"some_file.tif\")
  2. \n
  3. use the rasterio.merge to directly merge the files on disk:
  4. \n
\n
from pathlib import Path\nfrom rasterio import merge\n\nrasters_to_merge = [rasterio.open(filename) for filename in filenames]\ndestination = Path(\"merged_raster.tif\")\nmerge.merge(rasters_to_merge, dst_path=destination)\n\n
\n

My question (finally), is there a native way in xarray to perform the same operations? Thanks

","upvoteCount":1,"answerCount":1,"acceptedAnswer":{"@type":"Answer","text":"
\n

is there a native way in xarray to perform the same operations?

\n
\n

Not in a memory efficient way presently. The solution of writing to disk is a great way for you to merge larger datasets in a memory efficient way.

","upvoteCount":1,"url":"https://github.com/corteva/rioxarray/discussions/827#discussioncomment-11189893"}}}

merging very large datasets #827

Answered by snowman2
a2nalea asked this question in Q&A
Nov 8, 2024 · 1 comments · 2 replies
Discussion options

You must be logged in to vote

is there a native way in xarray to perform the same operations?

Not in a memory efficient way presently. The solution of writing to disk is a great way for you to merge larger datasets in a memory efficient way.

Replies: 1 comment 2 replies

Comment options

You must be logged in to vote
2 replies
@a2nalea
Comment options

@snowman2
Comment options

Answer selected by a2nalea
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants