Joe's Page

Baby Songs

Thu, 09 Jul 2020 00:00:00 +0000

Paula and I are expecting a baby girl to arrive sometime this month! Here’s a list of jams we’ve had fun rocking out to throughout the pregnancy. What are some of your favorites? Hope you enjoy!

I Got the Feelin’, James Brown

Baby, baby, baby
Baby, baby, baby
Baby, baby, baby
Baby, baby

Baby, I Love Your Way, Peter Frampton

Ooh, baby, I love your way
Wanna tell you I love your way
Wanna be with you night and day

Push It, Salt-N-Pepa

Oooh, baby, baby
Baby, baby

Baby Baby, Amy Grant

Baby, baby
I’m taken with the notion
To love you with the sweetest of devotion

Where Did Our Love Go, The Supremes

Baby, baby
Baby don’t leave me
Ooh, please don’t leave me
All by myself

Ooh Baby Baby, Smokey Robinson

Ooh oo-ooh baby baby
Ooh oo-ooh baby baby

Baby, I Love You, Ramones

Baby, I love you
Come on baby
Baby, I love you
Baby I love, I love only you

Yet Another Blender Hillshade Tutorial

Sun, 07 Apr 2019 00:00:00 +0000

Note: I acknowledge that this post could use more/better images and perhaps even some GIF’s or a video…but I wanted to just get it out there. I’ll try to make it better over time. Hit me up on Twitter or send me an email with any questions

Intro

After ogling at the cartographic works of Daniel Huffman, Andy Woodruff, Scott Reinhard and others - I wanted to take my stab at making one of those gorgeous Blender hillshades. I think I fell down this 🐰🕳via Andy’s post “Relief mapping in 10 easy steps!“…I don’t think I’ve even finished all 10 steps yet 😂 but it’s all been a good excuse to cut my teeth learning Blender, practice cartography with QGIS, and simply use GDAL 😍

For me, the point of all of this is purely cartographic, artistic, and hobby. Plus, one decent side effect of blogging is that you have a place to dump all of your notes for reference later. Yeah, the outputs probably won’t look as good as those aforementioned legends but from what I’ve seen so far, it certainly looks better than what I was “raised on”, when I was first getting started doing GIS … a lot better. So if this sounds like your jam, hopefully something noted here will help

example: gdaldem hillshade input-dem output-hillshade

same area: but rendered in Blender

This tutorial

Andy points to Daniel’s great series. But in all honesty, I got hung up on a step in The Plane section - so I googled around more and found Matt’s blog and finally Morgan’s post. Morgan’s notes allowed me to get some decent results by using the BlenderGIS plugin - and I decided to start documenting the steps to be able to show others. But I was able to come back and realize that one weird trick which I was missing for Daniel’s flow

Data & Setup

USGS’s TNM is solid for accessing DEM’s. Get some data to work with from here - or wherever. The 30-Meter SRTM Tile Downloader has some good stuff too (just register for a username/password first)

brew cask install blender 2.79b is the current release at the time this was written. By the way, I’ve used a Mac for this, so some assumptions will be made
make sure you’re on Cycles renderer And set your profile to default to this. Daniel covers this is Blender Basics
you will probably want to have QGIS around for checking DEM properties such as dimensions and pixel ranges - and to view your results. The Lutra Installer has been working 👌for me & I keep an OSGeo Homebrew version installed too

That One Weird Trick

Honestly, it took me a few days to realize this one ¯\_(ツ)_/¯ and at this point, I might as well point you back to Daniel’s tutorial. And while you’re at it, be sure to check out the rest of his site and support professional cartographers and artists like he and Scott. These people are true professionals who make some of their living with this work - donate to them or buy from them, if you can

Find the minimum and maximum elevation values of the source DEM, -16/3527 here, then:
gdal_translate -of GTiff -ot UInt16 -scale -16 3527 0 65535 N14W092.hgt dem-scale.tif

Why?

Our final output target is going to be a 16-bit unsigned integer TIFF. This means that each pixel can hold a number between 0 and 65,535

You’ll need to stretch your elevation range to cover the span from 0 to 65,535

source

Combining with topographic maps

This has been my playbook for obtaining old topos from topoView and combining them with DEM’s from TNM (mentioned above). I keep a notes.txt file with the steps below in a directory with my working files for organization. Shoutout 🗣to the QGIS team for awesome features like temporary scratch layers, better editing and snapping tools, and the Shape Digitizing Toolbar

Get the SRS info of the topo
- gdalsrsinfo CA_Dunsmuir_299340_1935_125000_geo.tif
- ☝️will output a PROJ.4 value like: +proj=poly +lat_0=0 +lon_0=-122.25 +x_0=0 +y_0=0 +datum=NAD27 +units=m +no_defs …log this somewhere
create a .geojson extent around the topo in QGIS
- add DEM first to set projection
- then add the topo .tif
- New Temporary Scratch Layer, geojson/polygon
- Digitize inside topo collar - around topo content. Save as topo-extent.geojson in working folder
warp & clip the DEM to the extent of the topo and into its projection
- gdalwarp -s_srs EPSG:4269 -t_srs '+proj=poly +lat_0=0 +lon_0=-122.25 +x_0=0 +y_0=0 +datum=NAD27 +units=m +no_defs' -r bilinear -cutline topo-extent.geojson -crop_to_cutline n42w123/imgn42w123_13.img dem.tif
add DEM to QGIS and get low/high elevations for next step
gdal_translate -of GTiff -ot UInt16 -scale 148.194 2525.06 0 65535 dem.tif dem-scale.tif
run orig topo tif through gdal_translate if you get something like oops.tif from the original topo
- gdal_translate CA_Dunsmuir_299340_1935_125000_geo.tif shasta.tif
get extent of topo and extend the DEM. You can do this with GDAL, but I usually use QGIS. Layer Properties > Information
- gdalwarp -te -25858.8718268426709983 4536063.6242974959313869 26708.5448398239932430 4598949.7909641629084945 dem-scale.tif dem-scaled.tif

note the X,Y dimensions of the topo/DEM

 Width
 4967
 Height
 5942

 For Blender scale/orthographic scale
 5942/4967 = 1.196295550634186
 1.196295550634186 * 2 = 2.392591101268371

Follow Daniel’s steps

Extra notes

Satellite imagery

Bart has a great post titled, “Photo-realistic shaded relief using Blender” 👍. I like the use of the cloudless Sentinel imagery and the Python wms-downloader

Shadows

The GIS Whisperer tweeted something neat on shadows, “Enhancing terrain cartography with natural shadows” which I tested a bit too. You can experiment with adding this on top to enhance/extend the Blender shadows

General tips and notes

Here are a few tips imma keep around for easy copy & paste. They’re not very well organized and some of them ended up not being required for the final workflows…going to note them anyway, for now:

basic reprojection. For a while I was using UTM as a common projection for the DEM’s, topos, and shadow layers I was using.

gdalwarp -s_srs EPSG:4269 -t_srs EPSG:26912 -r bilinear grdn37w111_13/hdr.adf dem-utm12n.tif

scale it. Here’s that one weird trick, again

gdal_translate -of GTiff -ot UInt16 -scale 1300.43 2465.47 0 65535 dem-utm12n.tif dem-utm12n-scale.tif

Using an extent polygon for trimming warp artifacts (black collar/no data/etc)

gdalwarp -cutline topo-extent.geojson -crop_to_cutline dem-utm12n-scale.tif dem-utm12n-crop.tif

export a worldfile for a dataset with known geobounds to use with a blender render which won’t have any

gdalwarp -co "TFW=YES" las-cruces-blender-dem.tif deleteme.tif
rm deleteme.tif
mv deleteme.tfw las-cruces-blender-hillshade.tfw

crop the hillshade. Say you want to digitize (QGIS) or otherwise produce (GDAL) an extent polygon to use for clipping your Blender hillshade. This is that same file from above with a companion .tfw, so it’ll produce a file with easy georeferenceability for carto hacking in QGIS

gdalwarp -s_srs EPSG:26913 -t_srs EPSG:26913 -cutline extent-topo.geojson -crop_to_cutline las-cruces-hillshade.tif las-cruces-hillshade-crop.tif

when reprojecting, -r bilinear resampling has worked best for me so far. -tr 5 5 if you need to scale the cell size/resolution

gdalwarp -s_srs EPSG:4269 -t_srs EPSG:26913 -r bilinear -tr 5 5 -cutline extent-shadow.geojson -crop_to_cutline -of GTiff USGS_NED_13_n33w107_IMG.img las-cruces-shadow-dem.tif

convert NAD27 Polyconic to UTM. Some of the old topos were in the projection. Also, -dstnodata 0 -dstalpha is handy if you want to blank out black collars/no data artifacts from warping

gdalwarp -t_srs EPSG:26912 -r bilinear -dstnodata 0 -dstalpha AZ_Marsh\ Pass_315539_1883_250000_geo.tif marshpass-utm12n.tif

Frittata

Thu, 03 Jan 2019 00:00:00 +0000

I had never made a frittata until recently, but wow, I’m glad I did. It’s turned out to be a hit at our house. They’re easy to make, you can throw anything in/on them, and cleanup’s not too bad. You can eat a piece warm or cold, as a side or a main, throw it on a piece of toast, add whatever sauce(s) you want - so much versatility.

View this post on Instagram

Got into the frittata game finally. Made my first one a couple months ago and have been practicing. Doing them baked-style...will have to try in cast iron sometime.

A post shared by joe larson (@jlarso) on Jan 2, 2019 at 1:58pm PST

Not sure how I found this recipe - it was likely the first thing to turn up in a search, but it’s proven to be a good guide for the half dozen or so I’ve made recently. Note: I’ve been doing my frittata in a 9” X 9” pyrex and entirely in the oven. I went with a square dish instead of a rectangle so the baking would hopefully be relatively consistent throughout. I’m sure I’ll get around to cooking them in cast iron - starting on the stove, then moving into the oven - but that’ll be another time. Here are the basics which have worked for me:

Ingredients

Required/most basic

6 large eggs
1/4 cup heavy cream
1 teaspoon salt
half or more of an onion, sauteed in a lil olive oil
2 cups or more spinach, sauteed as the onions are almost finished

Optional

1-2 cloves minced, sauteed garlic
1 cup shredded cheese. So far we’ve tried Jack and Parmesan (maybe save some or add more to the top at the end)
~3 asparagus stalks, chopped (asparagus and eggs go together so well)
1-2 small, cooked & finely sliced/chopped potatoes
chopped bacon
sun dried tomatoes
your favorite other ingredient or leftover!

Instructions

Heat oven 400F or do this later…because honestly, sometimes prep takes a while!
Mix eggs, cream, salt
Prep the goodness that you’re going to add…do your chopping, sauteing, grating, etc.
Butter the pyrex
Add “the goodness” into the egg mixture and mix more
Pour into buttered pyrex & toss it in the oven.
After 15-20 minutes, things should start looking good. Might take another 5+ mins …
For bonus points you can add more cheese near the end and finish with a broil

Holiday Letter

Wed, 12 Dec 2018 00:00:00 +0000

Hi, it’s been a while. But there’s movement in this little corner of the internet - another few lines in this log. I thought a good excuse to dust off this blog would be to get some ideas “down on paper” for an upcoming holiday letter to friends and family. Plus some coworkers who actually consistently blog have inspired me a bit lately. So, 2018 - here are some highlights:

February - Tahoe, CA w/ Sonia & Dauto; Paula’s parents came to visit and we took a trip to South Lake with my sister Megan and her boyfriend Travis. Good times indeed!
February/March - Camp Roberts, CA; went to a FX event representing SNI/Fulcrum Community - saw some awesome experiments, as usual.
March - Adobe Springs planting; I’ve been getting involved with the ALPS Native Tree Committee after seeing one of their signs near our house. It’s been a great experience participating in their volunteer events and having another excuse to use Fulcrum.
April - St. Pete; spring All Hands was pretty crazy with the growth that Spatial Networks is going through. Met a lot of new faces and laid the foundation for some good working relationships.
May - San Francisco, CA w/ Pablo & Camila; Paula’s cousins came to visit and they’re some of our favorite people in the universe. We took them to their first SF Giants game - which was very entertaining and the home team walked away with the W.

View this post on Instagram

I can’t remember how long it’s been since I’ve walked on this bridge. We did today and wow - it’s awesome. Good times with Pablo and Camila!

A post shared by joe larson (@jlarso) on May 20, 2018 at 11:01pm PDT

June - Firestone Invitational 🍺
June - Seinfeld; Paula and I went to see Jerry - it’s not too often that bigger events come through our small part of the world. Great show!
June - Tahoe again; Tony’s bachelor party

View this post on Instagram

Sometimes you just gotta post a mustache selfie after a polar bear plunge in Lake Tahoe. Water was 👌and stache is 👋

A post shared by joe larson (@jlarso) on Jun 27, 2018 at 4:00pm PDT

June/July - World Cup ⚽️
July - Beach Season ⛱
August - More Beach Season 🏖
August/September - Africa; Nairobi, Kenya…Dar es Salaam & Zanzibar, Tanzania for FOSS4G 2018 I had the opportunity to travel to East Africa for work and had a simply wonderful time. The people, food, weather, and culture were all fantastic - only bummer are the long flights!

View this post on Instagram

Made a new buddy leaving the Dar es Salaam airport

A post shared by joe larson (@jlarso) on Aug 28, 2018 at 5:36am PDT

September - Dallas; Tony’s wedding …got to have a 🌮with Ry.
October - St. Pete; fall All Hands saw work relationships strengthened even further - and I always enjoy St. Pete/Tampa!
November - Firestone Anniversary 🍻
December - Three Bridges Oak Preserve work day; remapped, replanted, and refreshed the mitigation plantings near the trailhead.
December - here we are!

I’m not going to commit to anything forthcoming - but it was nice to log a few lines again. I imagine I’ve omitted some highlights & I certainly have a whole lot to share since my last post over 2 years ago - like a new job … a new house, some new things I’m trying to learn, and other adventures. But we’ll start with this 👋hope you’re well.

My Local OpenStreetMap Story

Tue, 17 May 2016 00:00:00 +0000

tl;dr I worked with interns to make a lot of edits to OpenStreetMap at my previous job - a County/State Government. OSM wins, the Organization wins, the interns win - there are tons of benefits for many. Takeaway: teach others to edit OSM.

feed them for a lifetime

On a couple occasions Coleman has encouraged me to write about our accomplishments at my former job, where we leveraged power in numbers, to ferociously edit OpenStreetMap (OSM) in our local area. Aaand he’s right, I should! I’m going to jot this down before he reminds me again 😜.

My background in OSM began around the time I ‘cyberstalked’ @wonderchook on 6 May 2010 (thank you Twitter DM’s) somewhere near Monterey, California. I sat down with her for a beer and walked away inspired…can’t recall much that we talked about but I know it was enough of a catalyst to begin getting into OSM.

I live in a rural county in California, San Luis Obispo, affectionately known as SLO. It is significant in size (area) and has only a handful of population centers - the largest city is less than 50,000 people and the entire county has less than 300,000. I used to work for a full service all risk fire department - CAL FIRE, as a GIS Analyst. Also in SLO is Cal Poly State University, the reason I came to the area - as well as why many other students do - which we hire(d) as interns for CAL FIRE San Luis. Being a ‘rural’ area and working for an emergency response organization means that I/we needed to provide good data for a quick response to people that need help, sometimes in far away and/or obscure areas.

One of the things you do as a GIS Analyst is create data, specifically geographic databases/datasets. Something that commonly happens with GIS Analysts is that the data they create either becomes silo-ed, gets stale, or even is lost/destroyed - somehow, someway. During my first few years in that role, I experienced all of those possibilities. I learned that OSM could be a more permanent place for some of the data we were creating in our GIS lab. I was lucky enough to have an awesome supervisor who let me try things that were outside-of-the-box ideas and not traditional in our line of work - and we began working on OSM.

getting hungry

I guess you could say it all began with our lab wanting to collaboratively edit GIS data. We originally considered: pgVersion and zigGIS - from what I can recall. You may have guessed, our shop didn’t have a robust budget to afford ArcGIS for Server. As a State/County government agency, we were provided a few licenses of ArcMap but not enough for our entire team (which ranged from 5-10+ interns at a time) so we used QGIS and other FOSS4G software to fill in the gaps. But enabling collaborative editing of data in your own office is one thing - but what about opening up contributions from others outside of your office, organization, or even your country? BTW, those aforementioned approaches never really panned out for collaborative editing, until…

Enter OpenStreetMap: By contributing to OSM, our lab could concurrently edit the database while also benefitting from the crowd-sourced efforts of others around the world. Data can be created, manipulated, and extracted. The map and data we edit, in turn, are also used by dozens of significant companies and projects. We had found an ideal workflow for our lab to edit simultaneously, with a low barrier to entry (cost), which also benefitted from the power of crowd-sourcing, and the ability to get the data back out!

we feast

Source

Now with an idea on how we would proceed, we turned the interns loose. Our first project: building footprints. This project would provide multiple benefits to various users/cooperators.

The OpenStreetMap project would benefit by us enriching the map.
Our job in the GIS lab was to support the firefighters of CAL FIRE/San Luis Obispo County Fire; providing them with increased Situational Awareness during fire, medical, or rescue responses by having detailed maps.
The data we create is immediately viewable on the https://openstreetmap.org map.
The data becomes available to the variety of popular services that incorporate OSM data. For example: MAPS.ME was very popular with our field users because of its offline & routing capabilities.
The data can be extracted from OSM and converted into various formats for downstream uses/needs. For example: converting and sharing the data in Esri’s File Geodatabase with our local County cooperators.

We were blessed by the work of the Humanitarian OpenStreetMap Team for giving us the OSM Tasking Manager. This allowed us to work together in an organized and systematic way while adding our 150,000+ features across 3,300 square miles. It was also a good exercise for me in learning Ubuntu/Linux and Amazon EC2.

On a daily basis, this is how the lab would typically operate:

Wire up WhoDidIt to our Chrome RSS Feed Reader for quality control of the data in our area.
Review questionable edits with OSM History Viewer and/or osm deep history.
Field verify with GPS units and/then eventually Mapillary.
Utilize the power-editing tools in the JOSM editor. And use JOSM because it rhymes with Jaw-some, go Street Sharks!

digestifs

Sometimes I get a little sad 🙁 when I look at my How did you contribute to OpenStreetMap? UserLink, knowing that I’ve probably peaked with the amount of time I can give to editing.

But then I try to justify it by telling myself that I’ve taught and trained dozens of others to edit OSM and the efforts are still on-going in the lab - today! I don’t know ¯\_(ツ)_/¯ it is what it is. No ragrets 😉. I should probably just shut up and get back to mapping!

OSM Task Manager

Tue, 13 Oct 2015 00:00:00 +0000

I originally wrote this up for an intern who wanted to spin-up their own Task Manager…putting it here, in case others may benefit. Gist here for comments, etc. https://gist.github.com/oeon/a9dd7c4c73691030c135.

Login to https://aws.amazon.com/ > Go to the Console > click EC2.

Launch Instance > Select provided AMI in Quick Start section, for Ubuntu > t2.micro/free tier eligible will work fine…Click the Review and Launch button > accept defaults and click Launch button.

Create a new key pair, give key pair a name value, download key pair > Click Launch Instances after key pair has downloaded.

Click View Instances > In EC2 Dashboard, click Instances and first note your Public IP address (write it down), then scroll all the way to the right - click the launch-wizard link in the Security Groups section > In the Inbound tab > click Edit > Add Rule > Type=’All traffic’, Source=’Anywhere’, Save.

Connecting to your server, for Mac and Windows:

(For Mac/Unix) Open Terminal.app > alter the permissions of the key, something like chmod 0600 yourkey.pem.
Run something like ssh -i yourkey.pem ubuntu@12.345.67.89 in your Terminal.

(For Windows) Download Putty if needed. Navigate to where PuTTY is installed in All Programs and open the PuTTYgen utility. Load the .pem file and save as .ppk private key.

Putty > Connection > SSH > click on Auth section > load private key.
Click back on Session section and enter your IP address.
Save Session with a name for quick loading next time.

The login in username is ubuntu.

(Resume instructions for all operating systems) after sucessfully connecting to your server via SSH…

Update/upgrade server with sudo apt-get update, then sudo apt-get upgrade (accept by pressing Y). If reaching purple screen, accept default choice and press enter.

sudo apt-get install python-setuptools python-dev libpq-dev apache2 libapache2-mod-wsgi git postgresql-9.3-postgis-2.1 (accept by pressing Y).

Now follow setup steps documented at https://github.com/hotosm/osm-tasking-manager2.

(hint: right click to paste in PuTTY) enter commands line by line.

Create local.ini file with nano e.g. nano local.ini, change PASSWORD to your password.

Stop following the steps before the Launch the Application section.

Create an Apache config file. Copy the code from here: https://github.com/hotosm/osm-tasking-manager2#installation-as-a-mod_wsgi-application > Paste into a new file e.g. sudo nano /etc/apache2/sites-available/osmtm.conf be sure to alter the Location (line 13) parameter if you’re serving your app from a URL like http://myip.com/osmtm.

Create a file named OSMTM.wsgi in ~/osm-tasking-manager/env > Copy its contents from https://github.com/hotosm/osm-tasking-manager but change last line to match osm-tasking-manager2 not osm-tasking-manager.

Execute this series of commands:
sudo a2dissite 000-default
sudo a2ensite osmtm
sudo service apache2 reload

Open http://your-public-ip/osmtm e.g. http://12.345.67.89/osmtm in a web browser.
profit

Florence & Edison, 2014 Landsat

Tue, 09 Dec 2014 00:00:00 +0000

Continuing upon my last post using landsat-util and animating Landsat scenes - I took a look at path 42 row 34, where Florence and Lake Thomas Edison lie in the Sierra Nevadas of Central California.

I browsed EarthExplorer and copied the scenes I wanted.

path 42 row 34
LC80420342014015LGN00 01-15-14
LC80420342014047LGN00 02-16-14
LC80420342014079LGN00 03-20-14
LC80420342014095LGN00 04-05-14
LC80420342014143LGN00 05-23-14
LC80420342014175LGN00 06-24-14
LC80420342014207LGN00 07-26-14
LC80420342014239LGN00 08-27-14
LC80420342014255LGN00 09-12-14
LC80420342014287LGN00 10-14-14
LC80420342014319LGN00 11-15-14
LC80420342014335LGN00 12-01-14

To download all scenes at once, with landsat-util already installed - run:

landsat download LC80420342014015LGN00 && landsat download LC80420342014047LGN00 && landsat download LC80420342014079LGN00 && landsat download LC80420342014095LGN00 && landsat download LC80420342014143LGN00 && landsat download LC80420342014175LGN00 && landsat download LC80420342014207LGN00 && landsat download LC80420342014239LGN00 && landsat download LC80420342014255LGN00 && landsat download LC80420342014287LGN00 && landsat download LC80420342014319LGN00 && landsat download LC80420342014335LGN00

Process them all, takes a while - even longer if you pan-sharpen:

landsat process ~/landsat/zip/LC80420342014015LGN00.tar.bz && landsat process ~/landsat/zip/LC80420342014047LGN00.tar.bz && landsat process ~/landsat/zip/LC80420342014079LGN00.tar.bz && landsat process ~/landsat/zip/LC80420342014095LGN00.tar.bz && landsat process ~/landsat/zip/LC80420342014143LGN00.tar.bz && landsat process ~/landsat/zip/LC80420342014175LGN00.tar.bz && landsat process ~/landsat/zip/LC80420342014207LGN00.tar.bz && landsat process ~/landsat/zip/LC80420342014239LGN00.tar.bz && landsat process ~/landsat/zip/LC80420342014255LGN00.tar.bz && landsat process ~/landsat/zip/LC80420342014287LGN00.tar.bz && landsat process ~/landsat/zip/LC80420342014319LGN00.tar.bz && landsat process ~/landsat/zip/LC80420342014335LGN00.tar.bz

You could make those bash scripts and run both like ./download.sh && ./process.sh.

Next is the clip, label and gif combo. I run these from the processed folder that landsat-util created.

clip.sh

#!/bin/bash
mkdir clip
for i in $(find . -name "*.TIF" -type f)
do
  dir="$(dirname ${i#./})"
  gdal_translate -projwin -13264761 4497519 -13225720 4470697 -of GTiff $i clip/$dir"-clip.tif"
done

Of course you’ll want your own clip area. BYO bbox. Bring your own projwin. Maybe you’ll use QGIS like me, if so - be sure to swap the uly & lry coordinates.

label.sh

#!/bin/bash
mkdir gif
mogrify -format gif clip/*.tif
convert clip/LC80420342014015LGN00-clip.gif -fill rgba\(255,255,255,0.6\) -gravity NorthWest -pointsize 65 -annotate +5+5 'January' gif/01jan.gif
convert clip/LC80420342014047LGN00-clip.gif -fill rgba\(255,255,255,0.6\) -gravity NorthWest -pointsize 65 -annotate +5+5 'February' gif/02feb.gif
convert clip/LC80420342014079LGN00-clip.gif -fill rgba\(255,255,255,0.6\) -gravity NorthWest -pointsize 65 -annotate +5+5 'March' gif/03mar.gif
convert clip/LC80420342014095LGN00-clip.gif -fill rgba\(255,255,255,0.6\) -gravity NorthWest -pointsize 65 -annotate +5+5 'April' gif/04apr.gif
convert clip/LC80420342014143LGN00-clip.gif -fill rgba\(255,255,255,0.6\) -gravity NorthWest -pointsize 65 -annotate +5+5 'May' gif/05may.gif
convert clip/LC80420342014175LGN00-clip.gif -fill rgba\(255,255,255,0.6\) -gravity NorthWest -pointsize 65 -annotate +5+5 'June' gif/06jun.gif
convert clip/LC80420342014207LGN00-clip.gif -fill rgba\(255,255,255,0.6\) -gravity NorthWest -pointsize 65 -annotate +5+5 'July' gif/07jul.gif
convert clip/LC80420342014239LGN00-clip.gif -fill rgba\(255,255,255,0.6\) -gravity NorthWest -pointsize 65 -annotate +5+5 'August' gif/08aug.gif
convert clip/LC80420342014255LGN00-clip.gif -fill rgba\(255,255,255,0.6\) -gravity NorthWest -pointsize 65 -annotate +5+5 'September' gif/09sep.gif
convert clip/LC80420342014287LGN00-clip.gif -fill rgba\(255,255,255,0.6\) -gravity NorthWest -pointsize 65 -annotate +5+5 'October' gif/10oct.gif
convert clip/LC80420342014319LGN00-clip.gif -fill rgba\(255,255,255,0.6\) -gravity NorthWest -pointsize 65 -annotate +5+5 'November' gif/11nov.gif
convert clip/LC80420342014335LGN00-clip.gif -fill rgba\(255,255,255,0.6\) -gravity NorthWest -pointsize 65 -annotate +5+5 'December' gif/12dec.gif

gif.sh

#!/bin/bash
gifsicle gif/*.gif > anim.gif
gifsicle --loop=0 --delay=150 --colors 256 gif/*.gif > anim.gif

All together now ./clip.sh && ./label.sh && ./gif.sh. Yes, this could be more efficient but who knows your skill level…you may even want to build your scripts in a spreadsheet.

Bonus: Lake Oroville, California (2014) pansharpened http://i.imgur.com/biRxZqG.gif

Landsat animation

Sun, 07 Dec 2014 00:00:00 +0000

2014 Lake Nacimiento in San Luis Obispo County - California

Inspired by: https://twitter.com/developmentseed/status/539831619000217600 I wanted to play with some Landsat data to see if I could do a time series animation for 2014 to see if Central California’s recent drought could be effectively visualized. You can see some subtle shrinking of the ‘Dragon Lake’ …which is not as dramatic as Folsom or Shasta reservoirs, but still apparent.

It took a bit of time downloading & processing (pan-sharpening took about 50-60 minutes on my MacBook pro) and space (10GB+ for 11 scenes). My process could definitely be more scripted/streamlined. But I had fun and learned a few things :)

using landsat-util

https://github.com/developmentseed/landsat-util

landsat process --pansharpen LC80430352014022LGN00.tar.gz

clip

rename first, e.g. 4310final-pan.TIF

#!/bin/bash
gdal_translate -projwin -13475810.0 4270507.0 -13455685.0 4256681.0 -of GTiff ~/landsat/processed/LC80430352014310LGN00/4310final-pan.TIF ~/landsat/processed/4310final-pan-clip.TIF
gdal_translate -projwin -13475810.0 4270507.0 -13455685.0 4256681.0 -of GTiff ~/landsat/processed/LC80430352014294LGN00/4294final-pan.TIF ~/landsat/processed/4294final-pan-clip.TIF
gdal_translate -projwin -13475810.0 4270507.0 -13455685.0 4256681.0 -of GTiff ~/landsat/processed/LC80430352014262LGN00/4262final-pan.TIF ~/landsat/processed/4262final-pan-clip.TIF
gdal_translate -projwin -13475810.0 4270507.0 -13455685.0 4256681.0 -of GTiff ~/landsat/processed/LC80430352014214LGN00/4214final-pan.TIF ~/landsat/processed/4214final-pan-clip.TIF
gdal_translate -projwin -13475810.0 4270507.0 -13455685.0 4256681.0 -of GTiff ~/landsat/processed/LC80430352014198LGN00/4198final-pan.TIF ~/landsat/processed/4198final-pan-clip.TIF
gdal_translate -projwin -13475810.0 4270507.0 -13455685.0 4256681.0 -of GTiff ~/landsat/processed/LC80430352014166LGN00/4166final-pan.TIF ~/landsat/processed/4166final-pan-clip.TIF
gdal_translate -projwin -13475810.0 4270507.0 -13455685.0 4256681.0 -of GTiff ~/landsat/processed/LC80430352014134LGN00/4134final-pan.TIF ~/landsat/processed/4134final-pan-clip.TIF
gdal_translate -projwin -13475810.0 4270507.0 -13455685.0 4256681.0 -of GTiff ~/landsat/processed/LC80430352014118LGN00/4118final-pan.TIF ~/landsat/processed/4118final-pan-clip.TIF
gdal_translate -projwin -13475810.0 4270507.0 -13455685.0 4256681.0 -of GTiff ~/landsat/processed/LC80430352014070LGN00/4070final-pan.TIF ~/landsat/processed/4070final-pan-clip.TIF
gdal_translate -projwin -13475810.0 4270507.0 -13455685.0 4256681.0 -of GTiff ~/landsat/processed/LC80430352014054LGN00/4054final-pan.TIF ~/landsat/processed/4054final-pan-clip.TIF
gdal_translate -projwin -13475810.0 4270507.0 -13455685.0 4256681.0 -of GTiff ~/landsat/processed/LC80430352014022LGN00/4022final-pan.TIF ~/landsat/processed/4022final-pan-clip.TIF

convert

mogrify -format gif *.TIF

label

#!/bin/bash
convert 4022final-pan-clip.gif -fill rgba\(255,255,255,0.5\) -gravity North -pointsize 65 -annotate +0+5 'January 22' gifify/01jan.gif
convert 4054final-pan-clip.gif -fill rgba\(255,255,255,0.5\) -gravity North -pointsize 65 -annotate +0+5 'February 23' gifify/02feb.gif
convert 4070final-pan-clip.gif -fill rgba\(255,255,255,0.5\) -gravity North -pointsize 65 -annotate +0+5 'March 11' gifify/03mar.gif
convert 4118final-pan-clip.gif -fill rgba\(255,255,255,0.5\) -gravity North -pointsize 65 -annotate +0+5 'April 28' gifify/04apr.gif
convert 4134final-pan-clip.gif -fill rgba\(255,255,255,0.5\) -gravity North -pointsize 65 -annotate +0+5 'May 14' gifify/05may.gif
convert 4166final-pan-clip.gif -fill rgba\(255,255,255,0.5\) -gravity North -pointsize 65 -annotate +0+5 'June 15' gifify/06jun.gif
convert 4198final-pan-clip.gif -fill rgba\(255,255,255,0.5\) -gravity North -pointsize 65 -annotate +0+5 'July 17' gifify/07jul.gif
convert 4214final-pan-clip.gif -fill rgba\(255,255,255,0.5\) -gravity North -pointsize 65 -annotate +0+5 'August 02' gifify/08aug.gif
convert 4262final-pan-clip.gif -fill rgba\(255,255,255,0.5\) -gravity North -pointsize 65 -annotate +0+5 'September 19' gifify/09sep.gif
convert 4294final-pan-clip.gif -fill rgba\(255,255,255,0.5\) -gravity North -pointsize 65 -annotate +0+5 'October 21' gifify/10oct.gif
convert 4310final-pan-clip.gif -fill rgba\(255,255,255,0.5\) -gravity North -pointsize 65 -annotate +0+5 'November 06' gifify/11nov.gif

gif-ify

#!/bin/bash
gifsicle *.gif > anim.gif
gifsicle --loop=0 --delay=150 --colors 256 *.gif > anim.gif

fin

notes

path 43 row 35
LC80430352014022LGN00 22-JAN-14
LC80430352014054LGN00 23-FEB-14
LC80430352014070LGN00 11-MAR-14
LC80430352014118LGN00 28-APR-14
LC80430352014134LGN00 14-MAY-14
LC80430352014166LGN00 15-JUN-14
LC80430352014198LGN00 17-JUL-14
LC80430352014214LGN00 02-AUG-14
LC80430352014262LGN00 19-SEP-14
LC80430352014294LGN00 21-OCT-14
LC80430352014310LGN00 06-NOV-14

Landsat Pan Sharpening And Processing

Tue, 17 Dec 2013 00:00:00 +0000

tl;dr MapBox has some nice material on using Landsat 8 imagery. Without previous experience processing Landsat data, I used stuff learned from four of their blog posts. Here are some of my notes, images and a map of some local, recent Landsat imagery…nothing crazy, just a little do-it-yourself red-green-blue, false-color and color-infrared.

Initial inspiration came from Charlie’s two posts this summer - Putting Landsat 8’s Bands to Work and Processing Landsat 8 Using Open-Source Tools. Pansharpening gets you 15 meter resolution and Chris wrote the main steps involved in pansharpening with Orfeo Toolbox and GDAL.

December 5, 2013 Salinas, CA. False-color combination of NIR, SWIR, and visible red, using bands 5, 6, and 4 + pansharpened, Landsat 8.

I used Mac OS X and was hoping to evoke otbcli from Terminal command line. Thinking I could install Orfeo via Homebrew - it seems at the writing of this post there are various issues with homebrewed Orfeo/Mountain Lion. otbcli is contained within Monteverdi toolbox and I could probably access the applications/utilities from there but I already had QGIS installed and they were available from /Applications/QGIS.app/Contents/MacOS/bin/ on my system, so I used those.

Upper left, Harmony Headlands prescribed burn just getting underway - note the strong offshore winds pushing the smoke southwest. Lower right, Estero rx-burn from November. Image date: December 5, 2013.

Chris’s post involves GeoEye imagery, the code is pretty much he same if you use Landsat except I used this snippet from Derek for converting 16-bit bands (UInt16) to Byte type gdal_translate -ot Byte -scale 0 65535 0 255 -a_nodata "0 0 0" pansharp.tif pansharp-scaled.tif (only parameter that’s different is the scale value was 3000).

Wine country, San Luis Obispo County. You can see hints of red as the grape vines lose their leaves.

I ended up using convert -monitor -sigmoidal-contrast 50x12% RGB.tif RGB-corrected.tif for my regular Red-Green-Blue. And convert -monitor -sigmoidal-contrast 33x12% pansharp_3857.tif pansharp_3857-bright.tif for my pansharpened stuff. For those keeping count of those four MapBox posts I mentioned, the last honorable mention is from Processing RapidEye Imagery in Minutes where I used “GDAL one last time to bundle the adjusted image data and the geographical information into a geotiff”. Again, jotted my steps here.

City of San Luis Obispo and Santa Margarita lake, color-infrared.

It does take a bit of tweaking the contrast and brightness but I was happy enough with these quick results. Next, I’d like to recreate the Burned Area Reflectance Classification workflow. Also, more learning about Band Math, bit depth and which tool / which task for weilding Orfeo/ImageMagick/GDAL/? would be good. Check out band combinations for Landsat 8 here.

Note: A wiseman (Charlie) advised a couple things - “Always be a little careful pansharpening false color. The pan band only covers the RGB range, so to the extent that the other bands don’t positively correlate with it, it’s theoretically possible for it to distort their values.” And also, ImageMagick’s -combine flag doesn’t work correctly without a -colorspace RGB flag accompanying it (for some users) .. so heads-up!

Thank you kind internet people who put material out there for people to follow. Without much experience using remote sensing imagery or tools and this is a great way to get your feet wet. Again, the map.

Esri TMS for OSM

Wed, 02 Oct 2013 00:00:00 +0000

I wanted to log my efforts connecting to an Esri ArcGIS Server (AGS) raster layer / TMS-like tile service from JOSM and iD OpenStreetMap Editors. I didn’t see much material out-there, specifically for AGS, so maybe someone else will find this useful.

Previously, I wrote about cutting tiles for consumption via TMS…one method to share that gdal2tiles output is via Amazon S3. For example, a US Standard (region) bucket named ‘tile-demo’ connection URL for iD is: https://s3.amazonaws.com/tile-demo/{z}/{x}/{-y}.png. Thank goodness one can pass -y instead of running a script to rename TMS layout to Google layout on 6 million tiles.

Another option you may encounter is tiles available from an ArcGIS tile cache (is that the right term?). Here’s how I got there, provided your source also publishes the imagery under an appropriate license & in web mercator, EPSG 3857.

REST resource URL: http://gis.yourcounty.ca.gov/arcgis/rest/services

for iD: http://gis.yourcounty.ca.gov/arcgis/rest/services/Aerials/2014_1_FT_AERIAL/MapServer/tile/{z}/{y}/{x}

for JOSM, use ‘zoom’ instead of ‘z’: http://gis.yourcounty.ca.gov/arcgis/rest/services/Aerials/2014_1_FT_AERIAL/MapServer/tile/{zoom}/{y}/{x}

I didn’t need to specify image filetype in the URL e.g. .jpg / .png

So, if you have something like that available in your area - and it’s better quality/resolution that Bing .. try out those custom sources templates as base imagery during your OpenStreetMap editing.

EDIT: Ian Dees replied, “Another option is to use the WMS endpoint that ships with all MapServer instances with JOSM’s WMS code. OSM US can also proxy.” see https://twitter.com/iandees/status/386150404737622016