Lidar is a technique for displaying the shape of the ground using pulses of laser light. The results have been widely used to discover Roman monuments, as they can process them to omit modern buildings, trees, etc. I have been interested in this ever since I discovered some Lidar images of the seabed showing the submerged ruins of the Roman fort of Walton Castle at Felixstowe.
Most of the United Kingdom has been surveyed using Lidar, and the resulting datasets are now freely available for download on the Environment Agency / DEFRA website. If you can download them, then you can pull them into a tool like QGIS, and turn the data into images. But this website is not well organised. I have never been able to work out how to download anything!
Partly this is because I used my Android mobile much of the time. Just don’t. You won’t be able to get it to work. Instead go to your trusty PC and open your browser. I did all this in Chrome.
[Update: 19 March 2024: The website has changed, so I have updated these instructions.]
1. Go to the Defra Home page, and search for LIDAR
Go to https://environment.data.gov.uk/, ignore all of it and put LIDAR in the search box. The next page gives you more options to search:
- Change the search term to “LIDAR Composite” (without quotes)
- Change “Organisation” to “Environment Agency”
- Change “Sort by” to “Alphabetical A-Z”
- Change “Results per page” to 100.
You currently (March 2024) get 14 results, which look like this. (Click on the image for a larger image) (You can get more results if you omit “composite” but none looked useful).
If you scroll down, you will find “LIDAR Composite … DSM – 1m” and “LIDAR Composite … DSM – 2m”, and LIDAR Cmposite … DTM – 1 m” etc.
The LIDAR Composite DTM and DSM materials are what you want, taken at various resolutions. DSM is the raw data. DTM removes surface objects like trees and houses.
2. Click on LIDAR Composite Digital Terrain Model (DTM) – 1m
This takes you to a waffle page. At the bottom are various links:
The “WMS” links do not work – any of them. On enquiry I was told that:
You need to open the WMS/WFS links within the GIS software in order for the data to load, please refer to the following FAQ:
Apparently “WMS” is is a way to get the datasets from the GIS tool directly. Not what I have in mind here at the moment.
The one you want is the “Download the Survey Data” which I have highlighted in red.
3. Click on “Open The Link”
This takes you to https://environment.data.gov.uk/survey, which after a pause builds the following inscrutable screen:
Ignore the “Download” box for now!
The bit you will need is the square. But… NOT YET! It is, in fact, a tool to draw an area on the map. We’ll use it in a moment.
4. First, zoom into the area that you want to look at
This bit is fairly obvious. Use the “+” icon to zoom, and drag the map around. Once you get far enough in, a grid will appear with references on it. If you know the reference, you can enter it in the search box, although I notice this sometimes does not work.
I’m using the area off Felixstowe, so I get to this.
Until you are zoomed in, you can’t do anything. You can only download datasets for small areas, you see. But this is probably enough.
5. Draw a polygon on the map of the area for which you want Lidar
Now at last you can click on the “Draw polygon” button! So long as you stay over the map, it is surrounded by a grey area with a yellow border. If you move off to another window, it cancels this, tho! But now you can draw. (This frankly can be pretty tricky too.)
- Hover over the map at one corner of wherever you want to draw, and click. A small white circle should appear. If it doesn’t, look back at the “Draw polygon” icon – it’s probably lost focus.
- Now move the mouse. Click again, and you’ll get another white circle, and a blue line back to the last corner. Here’s a screen grab, (although the cursor went everywhere!)
- Repeat until your area is right, and then click back on your first circle to close the area. At that point the “Get available tiles” button on the popup will return to a normal green colour, and you can use it.
It will now look like this:
Note my polygon on the map.
Now, at last, you have something you can download. So click on the “Get Available Tiles”. You will now get a new menu:
What this lists is the various different types of dataset. In fact it lists the lot, of all sizes and resolutions. Whatever you choose, you get the link in blue at the bottom.
But there is more. Notice how the scroll bar to the right shows that the window is scrollable? If you scroll down you get this!
The link is to a zip file. In Chrome, just click on it to download to the Downloads folder; in IE, right-click and choose “Save target as” in the usual way. Either way you will end up with a lidar_composite_dtm-2022-1-TM33nw.zip file on your PC.
In my case, clicking “Download all” had the same effect.
[Note: I have not updated the following notes since 2019, so I don’t know if these work. I believe the bit above is what causes people trouble!]
6. Unzip the dataset
How you use the dataset is a different question, but I will give you what I found out.
First, you need to unzip the dataset. I use 7Zip on my PC, and right-click, 7-Zip, and extract to folder. So…
That created a folder Bathy-Coastal-Multibeam-2013-TM33nw in that directory.
I’m more interested in the DTM 1 meter stuff, so I get a download of LIDAR-DTM-1M-TM33nw.zip, and unpack it to a folder LIDAR-DTM-1M-TM33nw.
Inside the new folder are a bunch of .asc files. These together make up the dataset.
Next, you need a GIS tool to view this stuff.
7. Import into QGIS
I found this very hard to do, but here’s some notes on what I did. I worked it out based on this tutorial for an older version: http://geophyswithsnuffler.blogspot.com/2015/11/processing-uk-environment-agency-lidar.html
First, I installed the latest version of QGIS from the download site, which for me was 3.4.5. Look for the “long term stable release” stuff, and ignore the rest. This installed fine, and created a folder on my desktop, labelled QGIS 3.4, and an icon, “GRASS GIS 7.6.0”. Now … do NOT try to start that icon. Instead drag it into the folder, and forget about it.
Next open the folder, and double-click on the QGIS Desktop icon, again ignoring the GRASS thing. This will open something you can work with.
Next, create a project by Project -> New. Then do Project -> Save, and choose a name for your QGZ file – I used my own name.
Next, you need to import the dataset. Raster -> Miscellaneous -> Merge brings up a daft dialog box headed “Merge”.
Click on the “…”, and you get another daft dialog box headed “Multiple Selection”. Click on “Add”, and browse into the folder LIDAR-DTM-1M-TM33nw.
Select all the files in the folder, and hit “Open”. They will all appear in the “Multiple Selection” box.
Now hit “OK”. You’ll be back at the Merge dialog box.
You’ll want to save the resulting .tif file, so under “Merged” there is “Save to temporary file” – hit the “…” next to that and choose “Save to file”, and then pick a name.
Your “Merge” dialog will now look like this:
Don’t twiddle anything else.
Now hit Run, and go and make a cup of coffee. It takes a while.
When it finishes, it will pop up “Algorithm ‘Merge’ finished”, and look like this:
Hit “close” to get rid of the dialog box. You now have some results.
You can use the mouse to drag it around, and zoom in. The results are likely to look… disappointing.
On the left side is a box “Layers”. If you right-click on “Merged”, and choose “Properties”, you get stuff that you can play with. Select “Symbology”, and you can change the “Render type”. You can change it to “Hillshade” (whatever that is), and hit “Apply” and you get more details:
But that’s as far as I could get.
However, it IS more than I knew before.
Bibliography
Some links that I found useful:
- http://apps.environment-agency.gov.uk/wiyby/151365.aspx – overview of the datasets
- https://www.gislounge.com/what-is-a-shapefile/ – the datasets are “shape files”
- https://www.gislounge.com/shapefile-viewers/ – possible viewers
- https://houseprices.io/lab/lidar/map – Easily the best way to view Lidar. Only works on Chrome tho. Based on the 1m DSM data. Actually better than anything I got from this!
Actually the interface described is quite intuitive for Remote Sensing software. This is how most download pages for RS data work
You did not actually download LiDAR, you downloaded a Digital Terrain Model (DTM) created based on the LiDAR. LiDAR is a collection of points, the returns from the scan. it contains some extra information that the DTM does not include such as reflectance ratio which can help you understand what the laser actually reflected from. Now to create the DTM (also known as DEM, digital elevation model) whoever created made some assumptions, such as which points are ground, something that per the ASPRS guidelines should be included in any downloadable layer, and they used some geostatistical interpolation method such as Kriging or IDW to create the DEM. Since I have never used said dataset I have no idea if they are right or wrong.
Should you wish to see the actual LiDAR point cloud, the format you should download is LAS or its compressed version LAZ. LiDAR software is expensive, QGIS is not proper LiDAR software but rather GIS software with LiDAR extensions. Merrick MARS costs $4,995 per license, so unless your company can afford it you will not use it. CloudCompare is an open source solution you can install so you can visualize and navigate the point cloud. Be also aware that because of the size of LiDAR datasets displaying them will test our computer’s ability: a general purpose PC that is fine for coding will struggle running LiDAR, a machine with a solid state drive and a lot of RAM is recommended
Thank you very much for this. It is all rather confusing to the layman, as I can witness, and more info is very helpful!
Following on from ikokki’s excellent point, the Point Cloud Library is also really good, if you want to do more advanced things like 3D semantic segmentation. It has some impressive commerical and academic backers and is free, but, you will probably have to compile it yourself and you need to know C++ to use it, though there are very good tutorials. http://pointclouds.org/ Also, unless you have a GPU, you are going to experience a lot of pain trying to do anything meaningful with las/laz formats, though you can certainly extract info from DSM and DTM using QGIS and a CPU.
Can I extract canopy layer from the digital terrain model or maybe from the surface terrain model?
Alas, I have no idea.
Open data indeed…. your tutorial is both useful and hilarious Roger
Thank you!
Thanks very much I found this really useful, you’re a star!
You’re very welcome!
It astounds me that GIS people think this is ‘intuitive’. It’s like they have no idea of how an actual intuitive webpage works. There is nothing intuitive about a box telling you to upload something (no clues as to what) when you go to a page called ‘download your data’. Environment.gov.uk seems to have been put together by a sadistic sociopath with a deep hatred of people who just want to see a dtm.
Awful, isn’t it? I would imagine it was put together for internal use, and then made public facing.
HI Roger. Well done on getting through this minefield of how to bring up even an image of the ground topography. I’m trying to create my local golf course for a golf simulator (TGC 2019) and trying to get LIDAR data is a nightmare. At least you’ve given me an outline of the ground which may get me further toward my goal.
After using’ layers ‘ the option of contours produces very good results which I can use.
Very glad if my notes were helpful. It is atrociously difficult to get into, isn’t it?
I downloaded the data set for my area but just got 5 files, a .TFW, .TIF, .TIF Html, a TIF>AUX and a GPKG…..