spatial sample e57

Please post all open source software related items here, eg MeshLab
jedfrechette
V.I.P Member
V.I.P Member
Posts: 987
Joined: Mon Jan 04, 2010 7:51 pm
Full Name: Jed Frechette
Company Details: Lidar Guys
Company Position Title: CEO and Lidar Supervisor
Country: USA
Linkedin Profile: Yes
Location: Albuquerque, NM
Has thanked: 15 times
Been thanked: 61 times
Contact:

Re: spatial sample e57

Post by jedfrechette »

MalteHC wrote: Thu Oct 08, 2020 6:42 am Would it be possible to do a script, wheere it also will create .jpg 360 pictures from each staiton?
I don't think this is currently possible. If the e57 Reader created row and column index attributes for each point than it could be done easily by renaming those attributes to X and Y and then exporting an image with the Raster Writer. I think this is a good use case so it would be worth asking about on either the PDAL mailing list or their Gitter forum.

We have the same need as you do, however, our approach is to use Houdini's PDG to do things like estimate normals and convert e57 scans to images. Since PDG is fundamentally a job scheduler it can also be used to kick off external processes like PDAL pipelines that perform additional work.
Jed
User avatar
smacl
Global Moderator
Global Moderator
Posts: 491
Joined: Tue Jan 25, 2011 5:12 pm
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 141 times
Been thanked: 151 times
Contact:

Re: spatial sample e57

Post by smacl »

jedfrechette wrote: Thu Oct 08, 2020 7:02 pm
MalteHC wrote: Thu Oct 08, 2020 6:42 am Would it be possible to do a script, wheere it also will create .jpg 360 pictures from each staiton?
I don't think this is currently possible. If the e57 Reader created row and column index attributes for each point than it could be done easily by renaming those attributes to X and Y and then exporting an image with the Raster Writer. I think this is a good use case so it would be worth asking about on either the PDAL mailing list or their Gitter forum.

We have the same need as you do, however, our approach is to use Houdini's PDG to do things like estimate normals and convert e57 scans to images. Since PDG is fundamentally a job scheduler it can also be used to kick off external processes like PDAL pipelines that perform additional work.
I've done this in the past in SCC but it is quite involved. For spherical panoramic images you need to compute horizontal and vertical angles between each point and the setup station and use these as your X,Y indices for the image. You then need to interpolate colour (and depth for 3d) for all the blank pixels. All doable, but unless you've got a lot of time on your hands I'd be recommending a multi-threaded CPU or GPU solution rather than scripts ;)

This is also the basis of colorization of point clouds using flat photography and can yield some great results, Video here from early 2018. My feeling is that going forward point clouds will be derived from a hybrid of laser scanned measurements and HD images, rather than just using images to colour scans. The rather excelling point clouds produced by NavVis show the advantages of this, in terms of optimal point density and excellent colour.
jedfrechette
V.I.P Member
V.I.P Member
Posts: 987
Joined: Mon Jan 04, 2010 7:51 pm
Full Name: Jed Frechette
Company Details: Lidar Guys
Company Position Title: CEO and Lidar Supervisor
Country: USA
Linkedin Profile: Yes
Location: Albuquerque, NM
Has thanked: 15 times
Been thanked: 61 times
Contact:

Re: spatial sample e57

Post by jedfrechette »

smacl wrote: Fri Oct 09, 2020 9:40 am I've done this in the past in SCC but it is quite involved. For spherical panoramic images you need to compute horizontal and vertical angles between each point and the setup station and use these as your X,Y indices for the image.
I guess it depends on your goal, but for our needs we don't actually care about making a uniform angle lat long image. In fact, what we end up with is more like a stereo pair with left and right scan halves stored next to each other in the same image. That simplification makes the process trivial and fast.

After loading a structured scan we have a point cloud with:

Code: Select all

P.x
P.y
P.z
P.row_index
P.col_index
[P.intensity]
[P.red]
[P.green]
[P.blue]
Therefore to convert to an image the code isn't much more than:

Code: Select all

for P in pt_cloud:
    P.u = P.col_index
    P.v = P.row_index
Which, as you say, is embarrassingly parallel. I'm not sure of exact timing, but our implementation is pretty quick. It's certainly not a major bottleneck. The only reason you couldn't do something similar with PDAL is that the e57 reader doesn't create row and column index attributes. That's probably a fairly small change, which is why I suggested bringing it up on the mailing list
Jed
User avatar
smacl
Global Moderator
Global Moderator
Posts: 491
Joined: Tue Jan 25, 2011 5:12 pm
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 141 times
Been thanked: 151 times
Contact:

Re: spatial sample e57

Post by smacl »

jedfrechette wrote: Fri Oct 09, 2020 4:15 pmThe only reason you couldn't do something similar with PDAL is that the e57 reader doesn't create row and column index attributes. That's probably a fairly small change, which is why I suggested bringing it up on the mailing list
I would have thought that to index terrestrial laser scan points by row and column, you'd have to lay them on a 2d grid of some kind. Easy enough in 3d space with voxels, or by mapping to plan, but row and column indices won't be unique in this case unless you discard or average points such that you're left with just one point per cell. If you do this, you'll likely end up with gross loss of data. If you index based on angles by setup, you can avoid this.
jedfrechette
V.I.P Member
V.I.P Member
Posts: 987
Joined: Mon Jan 04, 2010 7:51 pm
Full Name: Jed Frechette
Company Details: Lidar Guys
Company Position Title: CEO and Lidar Supervisor
Country: USA
Linkedin Profile: Yes
Location: Albuquerque, NM
Has thanked: 15 times
Been thanked: 61 times
Contact:

Re: spatial sample e57

Post by jedfrechette »

smacl wrote: Sat Oct 10, 2020 6:22 pm I would have thought that to index terrestrial laser scan points by row and column, you'd have to lay them on a 2d grid of some kind.
If you're talking about individual structured scans stored in an e57 file then they are already laid out on a 2D grid (no different than ptx files). rowIndex and columnIndex are attributes that already exist on the PointRecord, so the reader just needs to get their values, no need to mess around with interpolation. Quoting from the e57 spec:
rowIndex
type: Integer
required/optional: optional
The row number of point (zero-based). This is useful for data that is
stored in a regular grid. Shall be in the interval [0, 2^63).

columnIndex
type: Integer
required/optional: optional
The column number of point (zero-based). This is useful for data
that is stored in a regular grid. Shall be in the interval [0, 2^63).
Of course since so much of the e57 spec is optional there is no guarantee than an arbitrary e57 includes these attributes, but we've setup our pipeline to only publish and use e57 files that conform to our requirements.
Jed
User avatar
smacl
Global Moderator
Global Moderator
Posts: 491
Joined: Tue Jan 25, 2011 5:12 pm
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 141 times
Been thanked: 151 times
Contact:

Re: spatial sample e57

Post by smacl »

jedfrechette wrote: Sat Oct 10, 2020 7:20 pm
smacl wrote: Sat Oct 10, 2020 6:22 pm I would have thought that to index terrestrial laser scan points by row and column, you'd have to lay them on a 2d grid of some kind.
If you're talking about individual structured scans stored in an e57 file then they are already laid out on a 2D grid (no different than ptx files). rowIndex and columnIndex are attributes that already exist on the PointRecord, so the reader just needs to get their values, no need to mess around with interpolation. Quoting from the e57 spec:
rowIndex
type: Integer
required/optional: optional
The row number of point (zero-based). This is useful for data that is
stored in a regular grid. Shall be in the interval [0, 2^63).

columnIndex
type: Integer
required/optional: optional
The column number of point (zero-based). This is useful for data
that is stored in a regular grid. Shall be in the interval [0, 2^63).
Of course since so much of the e57 spec is optional there is no guarantee than an arbitrary e57 includes these attributes, but we've setup our pipeline to only publish and use e57 files that conform to our requirements.
I don't think this is the case. The only data likely to be already stored in a regular grid is data that has already been rasterized, e.g. data coming from a GeoTIFF, DEM or ARC/INFO ASCII GRID. These optional attributes wont be present in laser scanned data because that data won't be rasterized. I don't think this is an issue with anyone's implementation of the E57 format so much as how the data has been collected and processed. I don't think PTX files coming from terrestrial laser scanning will typically be laid out in row column format of points either, the ASCII text is records (e.g. elements of a particular point) in a row which line up to form columns but no more than that. The natural row / column format for structured terrestrial scan data is all points in the same scan line (horizontal angle) forming a column with a new column for each horizontal angle shift. Problem here is that rows (i.e. vertical angles) may not line up. If they did, you could make laser scanning data way more compact.
jedfrechette
V.I.P Member
V.I.P Member
Posts: 987
Joined: Mon Jan 04, 2010 7:51 pm
Full Name: Jed Frechette
Company Details: Lidar Guys
Company Position Title: CEO and Lidar Supervisor
Country: USA
Linkedin Profile: Yes
Location: Albuquerque, NM
Has thanked: 15 times
Been thanked: 61 times
Contact:

Re: spatial sample e57

Post by jedfrechette »

Hi Shane,

I’m probably not explaining very well so thought I would try again with some illustrations. To begin, let’s start with a single structured scan in an e57 file:

https://master.dl.sourceforge.net/proje ... on018.e57

If we inspect the xml portion we can see what point attributes are present.

Code: Select all

      <points type="CompressedVector" fileOffset="48" recordCount="4067815">
        <prototype type="Structure">
          <cartesianX type="ScaledInteger" minimum="-536870912" maximum="536870911" scale="9.9999999999999995e-007"/>
          <cartesianY type="ScaledInteger" minimum="-536870912" maximum="536870911" scale="9.9999999999999995e-007"/>
          <cartesianZ type="ScaledInteger" minimum="-536870912" maximum="536870911" scale="9.9999999999999995e-007"/>
          <intensity type="ScaledInteger" minimum="0" maximum="32767" scale="3.0518509475997192e-005"/>
          <colorRed type="Integer" minimum="0" maximum="255"/>
          <colorGreen type="Integer" minimum="0" maximum="255"/>
          <colorBlue type="Integer" minimum="0" maximum="255"/>
          <rowIndex type="Integer" minimum="0" maximum="2047"/>
          <columnIndex type="Integer" minimum="0" maximum="8191"/>
          <cartesianInvalidState type="Integer" minimum="0" maximum="2"/>
        </prototype>
        <codecs type="Vector" allowHeterogeneousChildren="1">
        </codecs>
      </points>
Note that this is a structured scan so both rowIndex and columnIndex are defined. If your data is in a ptx file the row and column indexes aren’t given explicitly, but the shape of the 2D array is given in the header and values for all cells must be given, in a specific order, in the body so you can calculate the indexes. This isn’t very efficient from a storage stand point because you need to store values even for points without returns. I’ve always seen those no data points filled with coordinates of 0, 0, 0. It hasn’t been touched in a long time, but here is some old code that demonstrates this:

https://sourceforge.net/p/tlspy/code/ci ... es/ptx.py

In any case, once you have the indexes for each point everything I describe below applies to structured scans in either e57 or ptx format.

If we load that e57 point cloud I linked above and inspect the attributes we can see that they are all present. Note that we’ve renamed and scaled some of the attributes to match our working software’s conventions. In particular columnIndex == uv[0] and rowIndex == uv[1].

pt_cloud_data.JPG

Since the uv (aka rowIndex, columnIndex) coordinates define the scan point positions on a regular 2D grid, it is trivial to create a mesh by connecting each scan point to it’s neighbors in uv space. The top pane is the mesh in 3D space and the bottom two panes show the same mesh in uv space with the lower right pane zoomed in. Note that the vertexes of the mesh are still the original scan points, we haven’t changed them in anyway, simply added faces connecting them.

mesh_uvs.JPG

Technically, we don’t even need to generate a mesh for the next steps, but it is a good way to visualize the connectivity. Since the uv coordinates are a mapping to a 2D coordinate system we can use them directly as pixel indexes and render out the scan as an image.

comp_pano.jpg

The points have multiple attributes so we aren’t restricted to just using color to create an image. We can render out images for any of the point attributes including the x, y, and z coordinates. That also means that if we render images for all the attributes we can invert the process and losslessly convert those images back in to scans.

pt_coords.jpg

Here is a full resolution panorama generated using this method. You have to look closely to see it in this example, but if you zoom in on the exact center of the image you can find a vertical seem that splits the pano in half. That’s because you’re actually looking at the left and right halves of the scan sitting next to each other and the seam in the image is where they would overlap in 3D space. The panorama I’ve created here is NOT equivalent to putting a virtual 360 camera at the scan position and rendering the view. Doing that would be even easier, but for our use case we’re most interested in maintaining the structure of the scans so doing a direct conversion is better.

Station018_Color.jpg

The specifics of our workflow are a little specialized, but there’s no reason you can’t take the same approach with other tools. The core part of a hypothetical PDAL pipeline might be:

Code: Select all

[
    {
        "type":"readers.e57",
        "filename":"Station018.e57"
    },
    {
        "type":"filters.assign",
        "value" : "X = columnIndex"
    },
    {
        "type":"filters.assign",
        "value" : "Y = rowIndex"
    },
    ... Some other steps to save as a raster ...
]
The main blocker to doing that right now is that the attributes columnIndex and rowIndex aren’t read by readers.e57. Adding them should be a pretty small pull/feature request, which is why I suggest asking on the PDAL mailing list.
You do not have the required permissions to view the files attached to this post.
Jed
Post Reply

Return to “Open Source Software”