Skip to content

Updates to notebooks to demonstrate spatial subsetting #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Mikejmnez wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

Updates to notebooks to demonstrate spatial subsetting #21

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
95b082a
improve auth with earthaccess
Mikejmnez Apr 2, 2026
b711a57
Update TEMPO NRT notebook to include subsetting
Mikejmnez Apr 3, 2026
b014548
limit number of URLs for Tempo NRT
Mikejmnez Apr 3, 2026
a9439ac
fix for loop
Mikejmnez Apr 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions binder/CallipsoL2_access.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,10 +101,14 @@
},
"outputs": [],
"source": [
"auth = earthaccess.login(strategy=\"netrc\", persist=True) # you will be promted to add your EDL credentials\n",
"from earthaccess.exceptions import LoginStrategyUnavailable\n",
"try:\n",
" auth = earthaccess.login(strategy=\"netrc\", persist=True) # you will be promted to add your EDL credentials\n",
"except LoginStrategyUnavailable:\n",
" auth = earthaccess.login(strategy=\"interactive\", persist=True)\n",
"\n",
"# pass Token Authorization to a new Session.\n",
"my_session = session=auth.get_session()"
"my_session = session=auth.get_session()\n"
]
},
{
Expand Down
8 changes: 6 additions & 2 deletions binder/DAYMET.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,14 @@
"metadata": {},
"outputs": [],
"source": [
"auth = earthaccess.login(strategy=\"netrc\", persist=True) # \n",
"from earthaccess.exceptions import LoginStrategyUnavailable\n",
"try:\n",
" auth = earthaccess.login(strategy=\"netrc\", persist=True) # you will be promted to add your EDL credentials\n",
"except LoginStrategyUnavailable:\n",
" auth = earthaccess.login(strategy=\"interactive\", persist=True)\n",
"\n",
"# pass Token Authorization to a new Session.\n",
"my_session = auth.get_session()"
"my_session = session=auth.get_session()\n"
]
},
{
Expand Down
232 changes: 200 additions & 32 deletions binder/Ecostress.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,17 @@
"# Accessing ECOSTRESS data"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c5b07318-af7b-4470-b02d-a0ead104b568",
"metadata": {},
"outputs": [],
"source": [
"%load_ext autoreload\n",
"%autoreload 2"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand Down Expand Up @@ -43,10 +54,14 @@
"metadata": {},
"outputs": [],
"source": [
"auth = earthaccess.login(strategy=\"netrc\", persist=True) # you will be promted to add your EDL credentials\n",
"from earthaccess.exceptions import LoginStrategyUnavailable\n",
"try:\n",
" auth = earthaccess.login(strategy=\"netrc\", persist=True) # you will be promted to add your EDL credentials\n",
"except LoginStrategyUnavailable:\n",
" auth = earthaccess.login(strategy=\"interactive\", persist=True)\n",
"\n",
"# pass Token Authorization to a new Session.\n",
"my_session = session=auth.get_session()"
"my_session = session=auth.get_session()\n"
]
},
{
Expand All @@ -68,7 +83,7 @@
"source": [
"ECOSTRESS_ccid = \"C2076114664-LPCLOUD\"\n",
"bounding_box = [-128.847656,41.112469,-107.050781,46.679594]\n",
"time_range = [dt.datetime(2025, 3, 1), dt.datetime(2025, 3, 14)]\n",
"time_range = [dt.datetime(2025, 3, 1), dt.datetime(2025, 3, 30)]\n",
"\n",
"\n",
"cmr_urls = get_cmr_urls(ccid=ECOSTRESS_ccid, bounding_box = bounding_box, limit=1000, time_range=time_range) # limit by default = 50\n",
Expand All @@ -91,21 +106,38 @@
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "7744670c-cc38-4f92-a6bc-65e640657e8f",
"metadata": {
"tags": []
},
"source": [
"### Subset data\n",
"\n",
"First, we would like to subset data based on the criteria:\n",
"\n",
"* Daylight data\n",
"* QAPercentCloudCover <30% \n",
"* and QAPercentGoodQuality> 70%\n",
"\n",
"\n",
"For that, we need to download only 3 variables:\n",
"\n",
"```python\n",
" [\"/L2 LSTE Metadata/QAPercentCloudCover\", \n",
" \"/L2 LSTE Metadata/QAPercentGoodQuality\",\n",
" \"/StandardMetadata/DayNightFlag\"]\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "024bd204-f36c-4249-acce-f36d25aeb736",
"metadata": {},
"outputs": [],
"source": [
"keep_vars = ['/SDS/LST',\n",
" \"/StandardMetadata/EastBoundingCoordinate\",\n",
" \"/StandardMetadata/SouthBoundingCoordinate\",\n",
" \"/StandardMetadata/NorthBoundingCoordinate\",\n",
" \"/StandardMetadata/DayNightFlag\",\n",
" \"/StandardMetadata/WestBoundingCoordinate\",\n",
"]"
"output_path = 'data/'"
]
},
{
Expand All @@ -123,57 +155,141 @@
{
"cell_type": "code",
"execution_count": null,
"id": "f6936bb4-ed79-49e2-b70f-1f1256a60b78",
"metadata": {},
"id": "5fb47fd7-0743-4723-979b-e97ce2068303",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"output_path = \"data/\""
"%%time\n",
"dap_to_netcdf(cmr_urls, session=my_session, output_path = output_path, \n",
" keep_variables=[\"/L2 LSTE Metadata/QAPercentCloudCover\", \n",
" \"/L2 LSTE Metadata/QAPercentGoodQuality\",\n",
" \"/StandardMetadata/DayNightFlag\"]\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "c6c4b77f-7fa2-43e3-aed4-940d8c5d183d",
"metadata": {},
"source": [
"## Inspect local files\n",
"\n",
"Data has been already downloaded, and we can further filter to identify remote\n",
"granules by\n",
"\n",
"* Daylight data\n",
"* QAPercentCloudCover <30% \n",
"* and QAPercentGoodQuality> 70%\n",
"\n",
"Then we will update our list of URLs to download from, based on the remote files that\n",
"satisfy the above criteria\n",
"\n",
"**NOTE**: The files cannot be aggregated!!\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5fb47fd7-0743-4723-979b-e97ce2068303",
"metadata": {
"tags": []
},
"id": "77875486-b7e4-4f92-8929-b817dbef5244",
"metadata": {},
"outputs": [],
"source": [
"%%time\n",
"dap_to_netcdf(cmr_urls, session=my_session, output_path = output_path, keep_variables=keep_vars)"
"final_urls = []\n",
"for i in range(len(cmr_urls)):\n",
" local_file = output_path+cmr_urls[i].split(\"/\")[-1]+\".nc4\"\n",
" dst = xr.open_datatree(local_file).load()\n",
" if dst['L2 LSTE Metadata/QAPercentCloudCover'].values < 30 and dst[\"L2 LSTE Metadata/QAPercentGoodQuality\"] > 70 and dst[\"/StandardMetadata/DayNightFlag\"] == 'Day':\n",
" final_urls.append(cmr_urls[i])\n",
"\n",
"print(\"Total remote granules to download: \", len(final_urls))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a2765385-8703-4536-93ad-b3d9b6daba57",
"metadata": {
"tags": []
},
"id": "555ee69e-3ab1-4f68-8171-02da058a654c",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "fabf994b-a1e7-4982-ac9b-046a04ef760d",
"metadata": {},
"source": [
"### Declare final variables to download\n",
"\n",
"At this point, it is crucial to remove any ECOSTRESS data downloaded into the\n",
"data_output directory. For that open a terminal and navigate to the data directory\n",
"\n",
"\n",
"```shell\n",
"cd data\n",
"rm ECOv002_L2*.nc4\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0f7d60fb-6bad-4a97-a51c-2a9682396ec8",
"metadata": {},
"outputs": [],
"source": [
"dt = xr.open_datatree(\"data/ECOv002_L2_LSTE_37723_006_20250302T070023_0713_01.nc4\")\n",
"dt"
"# define all variables of interest that our final download will have\n",
"\n",
"keep_vars = [\"/StandardMetadata/EastBoundingCoordinate\", \n",
" \"/StandardMetadata/SouthBoundingCoordinate\", \n",
" \"/StandardMetadata/NorthBoundingCoordinate\",\n",
" \"/StandardMetadata/WestBoundingCoordinate\",\n",
" \"/StandardMetadata/DayNightFlag\",\n",
" \"/StandardMetadata/ImagePixels\",\n",
" \"/StandardMetadata/ImagePixelSpacing\",\n",
" \"/StandardMetadata/ImageLines\",\n",
" \"/StandardMetadata/RangeBeginningDate\",\n",
" \"/StandardMetadata/RangeBeginningTime\",\n",
" \"/StandardMetadata/RangeEndingDate\",\n",
" \"/L2 LSTE Metadata/QAPercentCloudCover\",\n",
" \"/L2 LSTE Metadata/QAPercentGoodQuality\",\n",
" \"/SDS/QC\", \"/SDS/LST\",\n",
" ]"
]
},
{
"cell_type": "markdown",
"id": "41c429de-1bab-4a79-8fea-6055073bbf13",
"metadata": {},
"source": [
"### Data Download\n",
"\n",
"Do not forget to now use the final_urls list, which points to our granules of interest.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "30c713c6-cee4-4ba4-8b4c-1edb8ee09583",
"metadata": {
"tags": []
},
"id": "a17ec9bb-ecb4-4c9f-b198-32a484f1a225",
"metadata": {},
"outputs": [],
"source": [
"dt['SDS/LST'].isel(dim0=slice(0, 1000), dim1=slice(0, 1400)).plot()"
"%%time\n",
"dap_to_netcdf(final_urls, session=my_session, output_path = output_path, \n",
" keep_variables=keep_vars,\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "5c60ff18-746f-4df0-925e-542fb81007e1",
"metadata": {},
"source": [
"### Inspect local files\n",
"\n",
"Data has been already downloaded! \n",
"\n",
"**NOTE:** This datasets cannot be aggregated\n"
]
},
{
Expand All @@ -182,6 +298,58 @@
"id": "7b095687-800d-48c2-a1cc-50cd8f733fb0",
"metadata": {},
"outputs": [],
"source": [
"local_file = output_path+final_urls[0].split(\"/\")[-1]+\".nc4\"\n",
"dst = xr.open_datatree(local_file)\n",
"dst\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c3ce3ff9-2e1d-4f8a-a164-772dc9f6695a",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "330aac77-5748-4875-beb9-8a9564bca085",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "b9e25760-4361-48a3-9e50-e2a1a1164359",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "0128a980-ea83-48b4-ad0e-4e6eed0b26f2",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "54f32f9d-2731-4cc9-9685-1e63582afbd3",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "15276546-459e-4dc1-a7b8-5fc0140d24f2",
"metadata": {},
"outputs": [],
"source": []
}
],
Expand Down
8 changes: 6 additions & 2 deletions binder/MERRA2.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,10 +97,14 @@
},
"outputs": [],
"source": [
"auth = earthaccess.login(strategy=\"netrc\", persist=True) # you will be promted to add your EDL credentials\n",
"from earthaccess.exceptions import LoginStrategyUnavailable\n",
"try:\n",
" auth = earthaccess.login(strategy=\"netrc\", persist=True) # you will be promted to add your EDL credentials\n",
"except LoginStrategyUnavailable:\n",
" auth = earthaccess.login(strategy=\"interactive\", persist=True)\n",
"\n",
"# pass Token Authorization to a new Session.\n",
"my_session = auth.get_session()"
"my_session = session=auth.get_session()\n"
]
},
{
Expand Down
16 changes: 6 additions & 10 deletions binder/PACEL3_Chrol_a.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,14 +94,6 @@
"chlor_a_urls[:5]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8fce82b9-a48f-40f6-8fbd-a4be6c67473f",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "d8e45e61-896d-456e-9e0c-9e0b504b62da",
Expand All @@ -123,10 +115,14 @@
},
"outputs": [],
"source": [
"auth = earthaccess.login(strategy=\"netrc\", persist=True) # you will be promted to add your EDL credentials\n",
"from earthaccess.exceptions import LoginStrategyUnavailable\n",
"try:\n",
" auth = earthaccess.login(strategy=\"netrc\", persist=True) # you will be promted to add your EDL credentials\n",
"except LoginStrategyUnavailable:\n",
" auth = earthaccess.login(strategy=\"interactive\", persist=True)\n",
"\n",
"# pass Token Authorization to a new Session.\n",
"my_session = session=auth.get_session()"
"my_session = session=auth.get_session()\n"
]
},
{
Expand Down
Loading