It's Friday, July 15th 2022 and next week, Things That Caught My Attention will be on vacation. I'll be back on Monday 25th July, hopefully with some nice pictures of the Milky Way, too.
\nSo I'm busy looking at the James Webb Space Telescope's stunning images of the Carina Nebula, and this is how my brain works:
\nMe: Hey, this new laptop is supposed to have some kind of awesome display with better dynamic range and brightness (so Apple says), so is there a way in which this already awesome image of the Cosmic Cliffs could look better?
\nAnd the answer, in a roundabout way, is... maybe? The TIFF that I downloaded, the giant, hundred-plus megabyte one, isn't an HDR image, it's got a regular sRGB color profile and I can't easily tell that it's got more bits per pixel.
\nSo it's an entirely reasonable question when someone asks: well, can you get the raw data from JWST?
\nOf course you can. It's not like somebody's going to fund an ~$8 billion telescope and not make the data public, right? So off I go and start looking it up and it turns out JWST makes all of its data available from its AWS S3 buckets, of all places (I mean, where else?) and you can go and grab shell scripts that will download the data from the Webb First Images.
\n\n\nProtip: I've got a gigabit fiber connection that I'm wired into over ethernet and grabbing the Carina images (a good ~310GB) took about an hour, maybe?
\n
Anyway, you grab all that raw data and then you find out that sure, Photoshop and Affinity Photo can open the raw FITS files, but you also start learning about the JWST calibration pipeline and reading the Getting Started with JWST Data documentation and really, the software stack and management here looks super impressive! You end up learning about FITS Liberator, an open source, cross-platform application for processing FITS data and turning it into stuff you can muck about with and stack.
\nBut the short answer is: if you wanted to make HDR images of JWST data, given that the sensors record 16 bit raw data and then you've got to map all of that to "colors" then sure, knock yourself out.
\nIn the meantime, I learned that it's probably possible and I lazywebbily look forward to someone else doing it.
\nThat's it for today! It's been a super stressful week. See you in just over a week.
\nBest,
\nDan
\n","body_text":"0.0 Context Setting\n\nIt's Friday, July 15th 2022 and next week, Things That Caught My Attention will be on vacation. I'll be back on Monday 25th July, hopefully with some nice pictures of the Milky Way, too.\n\n1.0 Some Things That Caught My Attention\n\nRabbit hole\n\nSo I'm busy looking at the James Webb Space Telescope's\nstunning images of the Carina Nebula\n, and this is how my brain works:\n\nMe: Hey, this new laptop is supposed to have some kind of awesome display with better dynamic range and brightness (so Apple says), so is there a way in which this already awesome image of the Cosmic Cliffs could look\nbetter\n?\n\nAnd the answer, in a roundabout way, is... maybe? The TIFF that I downloaded, the giant, hundred-plus megabyte one, isn't an HDR image, it's got a regular sRGB color profile and I can't easily tell that it's got more bits per pixel.\n\nSo it's an entirely reasonable question when someone asks: well, can you get the raw data from JWST?\n\nOf course you can. It's not like somebody's going to fund an ~$8 billion telescope and not make the data public, right? So off I go and start looking it up and it turns out JWST makes all of its data available from its AWS S3 buckets, of all places (I mean, where else?) and you can go and grab\nshell scripts that will download the data from the Webb First Images\n.\n\nProtip: I've got a gigabit fiber connection that I'm wired into over ethernet and grabbing the Carina images (a good ~310GB) took about an hour, maybe?\n\nAnyway, you grab all that raw data and then you find out that sure, Photoshop and Affinity Photo can open the raw\nFITS\nfiles, but you also start learning about the\nJWST calibration pipeline\nand reading the\nGetting Started with JWST Data documentation\nand really, the software stack and management here looks super impressive! You end up learning about\nFITS Liberator\n, an open source, cross-platform application for processing FITS data and turning it into stuff you can muck about with and stack.\n\nBut the short answer is: if you wanted to make HDR images of JWST data, given that the sensors record 16 bit raw data and then you've got to map all of that to \"colors\" then sure, knock yourself out.\n\nIn the meantime, I learned that it's probably possible and I lazywebbily look forward to someone else doing it.\n\nThat's it for today! It's been a super stressful week. See you in just over a week.\n\nBest,\n\nDan","raw_format":"markdown","source":"app","summary":"This issue follows a characteristic rabbit hole: starting with admiration for the James Webb Space Telescope's Carina Nebula images, the author wonders whether they might look even better on a new high-dynamic-range laptop display, which leads to investigating whether raw JWST data is publicly available. What emerges is a deep dive into the surprisingly accessible infrastructure around JWST data—AWS S3 buckets, 310GB downloads, calibration pipelines, and open-source tools like FITS Liberator—revealing both the technical ecosystem that makes the data available and the question of whether someone might eventually process that raw 16-bit sensor data into true HDR images. The throughline is curiosity chasing itself: each answer spawns new questions that pull deeper into the technical weeds, ultimately settling on the comfortable realization that while it's theoretically possible, the author is content to wait for someone else to do the work.","reading_minutes":2,"links":[{"url":"https://webbtelescope.org/contents/media/images/2022/031/01G77PKB8NKR7S8Z6HBXMYATGJ","normalized_url":"https://webbtelescope.org/contents/media/images/2022/031/01G77PKB8NKR7S8Z6HBXMYATGJ","domain":"webbtelescope.org","anchor_text":"stunning images of the Carina Nebula","is_archive":0,"archive_url":null,"archive_service":null,"category":"external"},{"url":"https://outerspace.stsci.edu/display/MASTDATA/JWST+AWS+Bulk+Download+Scripts","normalized_url":"https://outerspace.stsci.edu/display/MASTDATA/JWST+AWS+Bulk+Download+Scripts","domain":"stsci.edu","anchor_text":"shell scripts that will download the data from the Webb First Images","is_archive":0,"archive_url":null,"archive_service":null,"category":"external"},{"url":"https://fits.gsfc.nasa.gov","normalized_url":"https://fits.gsfc.nasa.gov/","domain":"nasa.gov","anchor_text":"FITS","is_archive":0,"archive_url":null,"archive_service":null,"category":"external"},{"url":"https://github.com/spacetelescope/jwst","normalized_url":"https://github.com/spacetelescope/jwst","domain":"github.com","anchor_text":"JWST calibration pipeline","is_archive":0,"archive_url":null,"archive_service":null,"category":"external"},{"url":"https://jwst-docs.stsci.edu/getting-started-with-jwst-data","normalized_url":"https://jwst-docs.stsci.edu/getting-started-with-jwst-data","domain":"stsci.edu","anchor_text":"Getting Started with JWST Data documentation","is_archive":0,"archive_url":null,"archive_service":null,"category":"external"},{"url":"https://en.wikipedia.org/wiki/FITS_Liberator","normalized_url":"https://en.wikipedia.org/wiki/FITS_Liberator","domain":"wikipedia.org","anchor_text":"FITS Liberator","is_archive":0,"archive_url":null,"archive_service":null,"category":"external"}],"sections":[{"id":2420,"ord":0,"number":"0.0","heading":"Context Setting","level":1,"word_count":36},{"id":2421,"ord":1,"number":"1.0","heading":"Some Things That Caught My Attention","level":1,"word_count":6},{"id":2422,"ord":2,"number":null,"heading":"Rabbit hole","level":2,"word_count":388}]}