Big Data From Space: 8 Best Practices For Geospatial Development

In recent years, the data acquired by earth observation (EO) satellites has been growing exponentially. In addition to increasingly large number of satellites orbiting the Earth, the improved spatiotemporal resolution has played significant role towards building of
data volume. Therefore, the availability of big EO data is fuelling the development of data driven applications.

The most important aspects of any operational application are scalability and efficient data management and data processing. Keeping this in view along with the challenges in terms of data growth and diversity from imaging satellites, here we suggest some key strategic practices which are crucial for big geospatial data handling, processing and analysis:

1. Divide the main task into small chunks
2. Implement these chunks in a robust style
3. Try to avoid loopholes for manual work
   • Always keep in mind:
   • automation
   • robustness
   • transferability
   • scalability
   • reproducibility
4. Maintain proper documentation from the beginning
5. Keep the graphical flowchart (i.e., UML diagram) updated in order to better visualize the hierarchy and dependency structure of the proposed solution
6. In this era of big-geospatial-data, think about the bringing solution or software to the data not the other way round
7. Clear and well thought-out file/folder naming structure of input/output data is very important for bulk processing, tracing logs and error fixing
8. Maintenance of all necessary metadata and log-files is high recommended for big-geospatial-data processing and product development

These practices become more relevant and necessary when you are planned to move from a feasibility analysis to an operational near real-time service. These guidelines will eventually help you to maintain and operate at a large spatial scale with much more stability.