mirror of
https://github.com/google/sandboxed-api.git
synced 2024-03-22 13:11:30 +08:00
Project architecture redesign, coding style update
Put duplicated code inside the transaction, which is used by both tests and raster_to_gtiff Removed <filesystem> header, reimplemented one of its utilities for file checking Code style changes Replaced .data() with .c_str() for std::string Updated README to show how to build both GDAL and PROJ inside the build folder and how to link them to the sandbox
This commit is contained in:
Bohdan Tyshchenko
parent
48348d6aa5
commit
b1a1aef39e
|
|
@ -86,14 +86,35 @@ target_link_libraries(data_retriever
|
|||
libgdal
|
||||
)
|
||||
|
||||
add_library(utils STATIC
|
||||
utils.h utils.cc
|
||||
)
|
||||
|
||||
target_link_libraries(utils PUBLIC
|
||||
sapi::sapi
|
||||
sandbox2::temp_file
|
||||
sandbox2::fileops
|
||||
sandbox2::util
|
||||
)
|
||||
|
||||
add_library(gtiff_converter STATIC
|
||||
gtiff_converter.h gtiff_converter.cc
|
||||
)
|
||||
|
||||
target_link_libraries(gtiff_converter PUBLIC
|
||||
data_retriever
|
||||
gdal_sapi
|
||||
sapi::sapi
|
||||
)
|
||||
|
||||
add_executable(raster_to_gtiff
|
||||
raster_to_gtiff.cc
|
||||
)
|
||||
|
||||
target_link_libraries(raster_to_gtiff
|
||||
data_retriever
|
||||
gdal_sapi
|
||||
sapi::sapi
|
||||
utils
|
||||
gtiff_converter
|
||||
)
|
||||
|
||||
if (ENABLE_TESTS)
|
||||
|
|
@ -102,11 +123,9 @@ if (ENABLE_TESTS)
|
|||
|
||||
add_executable(tests tests.cc)
|
||||
target_link_libraries(tests PRIVATE
|
||||
gdal_sapi
|
||||
data_retriever
|
||||
sapi::sapi
|
||||
sandbox2::temp_file
|
||||
sandbox2::fileops
|
||||
gtiff_converter
|
||||
utils
|
||||
${GTEST_LIBRARIES}
|
||||
${GTEST_MAIN_LIBRARIES}
|
||||
)
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
|
||||
# GDAL Raster to GeoTIFF Workflow Sandbox
|
||||
This repository is an example of how Sandboxed API can be used with GDAL C Raster API to implement the creation of the GeoTIFF dataset inside the sandbox.
|
||||
|
||||
|
|
@ -17,25 +16,42 @@ Implemented workflow consists of a few steps:
|
|||
This project consists of a CMake file that shows how you can connect Sandboxed API and GDAL, a raster data parser using unsandboxed GDAL to generate sample input for the sandboxed workflow, sample sandbox policy that could work with GeoTIFF files without any violations, command-line utility that uses sandboxed GDAL to implement the workflow and GoogleTest unit tests to compare raster data of original datasets with the raster data of datasets that have been created inside the sandbox.
|
||||
|
||||
## Build GDAL sandbox
|
||||
To build a GDAL sandbox, it's required to have a static build of libgdal and libproj. Moreover, proj.db file path is required to be able to map it inside the sandbox and use it internally for some of the projections.
|
||||
Because GDAL doesn't use CMake or Bazel it's required to have a static build of libgdal and libproj. Moreover, proj.db file path is required to be able to map it inside the sandbox and use it internally for some of the projections.
|
||||
|
||||
### Build GDAL and PROJ from sources
|
||||
To get the latest version of both GDAL and PROJ you will need to build them from sources.
|
||||
First, you should build PROJ with this [tutorial](https://proj.org/install.html#compilation-and-installation-from-source-code).
|
||||
After the installation, you should have a static build of libproj, remember the path as you will need to specify it later in CMake build.
|
||||
Then, get gdal sources using git submodules:
|
||||
To get the latest version of both GDAL and PROJ you will need to build them from sources. To make a clean installation you can use the build folder as an installation path, with this approach you won't affect any system files and will be able to easily delete everything later.
|
||||
First, you will need to build PROJ, which is used internally in the GDAL. You can't use the libproj-dev package because it contains an outdated version, while GDAL requires a more recent one.
|
||||
To [install PROJ from sources](https://proj.org/install.html#compilation-and-installation-from-source-code) you can do the following:
|
||||
```
|
||||
git submodule add https://github.com/OSGeo/gdal/
|
||||
git submodule update --init --recursive
|
||||
mkdir build && cd build
|
||||
wget https://download.osgeo.org/proj/proj-7.1.1.tar.gz
|
||||
tar xvzf proj-7.1.1.tar.gz
|
||||
mkdir proj_build
|
||||
cd proj-7.1.1
|
||||
./configure --prefix=/path/to/build/proj_build
|
||||
make -j8
|
||||
make install
|
||||
make check
|
||||
```
|
||||
After that you can go to the GDAL sources and make a static build of libgdal:
|
||||
The static version of libproj will be available at `proj_build/lib/libproj.a`.
|
||||
Then, you can start [GDAL installation](https://trac.osgeo.org/gdal/wiki/BuildingOnUnix):
|
||||
```
|
||||
cd build
|
||||
git clone https://github.com/OSGeo/gdal
|
||||
mkdir gdal_build
|
||||
cd gdal/gdal
|
||||
./configure --with_proj=/path/to/proj/
|
||||
make static-lib
|
||||
./configure --prefix=/path/to/build/gdal_build --with-proj=/path/to/build/proj_build
|
||||
make -j8
|
||||
make install
|
||||
```
|
||||
**Note**: On the `./configure` step you should specify the path to your proj library as a `--with-proj=` argument to make everything work correctly.
|
||||
### Build GDAL using dev-packages
|
||||
To verify that everything is installed correctly you can run gdalinfo utility.
|
||||
```
|
||||
cd ../../gdal_build/bin/
|
||||
./gdalinfo --version
|
||||
```
|
||||
The static version of libgdal will be available at `gdal_build/lib/libgdal.a`.
|
||||
You will need to specify paths to those static libraries as the CMake arguments to build the project. Also, the Sandboxed API generator needs `gdal.h` header, which is located at `build/gdal/gdal/gcore/gdal.h`.
|
||||
|
||||
### Build sandboxed GDAL
|
||||
To build the examples from this repository you can use CMake in the following way:
|
||||
```
|
||||
|
|
@ -56,12 +72,19 @@ cmake .. -G Ninja -DSAPI_ROOT=/path/to/sapi \
|
|||
-DLIBPROJ_PREFIX=/path/to/libproj_static_build \
|
||||
-DENABLE_TESTS=ON
|
||||
```
|
||||
After CMake build completed you can run `ninja` to build executables.
|
||||
After CMake build is completed, you can run `ninja` to build the executables.
|
||||
|
||||
## Examples
|
||||
Before running any of the examples, you need to specify the path to the `proj.db` using the environment variable.
|
||||
To do so, run `export PROJ_PATH=/path/to/proj.db`. Alternatively, if there is no such environment variable program will try to use the default path `/usr/local/share/proj/proj.db`.
|
||||
PROJ uses `proj.db` database to work correctly with different transformations and you will need to map it to the sandbox manually to be able to retrieve it in the restricted environment. After the installation you can find `proj.db` in `/path/to/proj/share/proj/proj.db`.
|
||||
|
||||
You can use environment variables to set path to proj:
|
||||
```
|
||||
export PROJ_PATH=/path/to/proj.db
|
||||
```
|
||||
The code will check this variable and if it represents a valid file it will be mounted inside the sandbox.
|
||||
Alternatively, if there is no such environment variable program will try to use the default path `/usr/local/share/proj/proj.db`.
|
||||
There is a simple command-line utility that takes path to the GeoTIFF file and absolute path to the output file as arguments, parses raster data from the input file and, re-creates the same GeoTIFF file (except some metadata) inside the sandbox.
|
||||
|
||||
You can run it in the following way:
|
||||
`./raster_to_gtiff path/to/input.tif /absolute/path/to/output.tif`
|
||||
After that, you can compare both files using the `gdalinfo` utility.
|
||||
|
|
|
|||
|
|
@ -12,12 +12,11 @@
|
|||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef GDAL_SANDBOX_H_
|
||||
#define GDAL_SANDBOX_H_
|
||||
#ifndef RASTER_TO_GTIFF_GDAL_SANDBOX_H_
|
||||
#define RASTER_TO_GTIFF_GDAL_SANDBOX_H_
|
||||
|
||||
#include <syscall.h>
|
||||
|
||||
#include <iostream>
|
||||
#include <string>
|
||||
|
||||
#include "sandboxed_api/sandbox2/util/fileops.h"
|
||||
|
|
@ -70,4 +69,4 @@ class GdalSapiSandbox : public gdalSandbox {
|
|||
|
||||
} // namespace gdal::sandbox
|
||||
|
||||
#endif // GDAL_SANDBOX_H_
|
||||
#endif // RASTER_TO_GTIFF_GDAL_SANDBOX_H_
|
||||
|
|
|
|||
|
|
@ -14,12 +14,11 @@
|
|||
|
||||
#include "get_raster_data.h" // NOLINT(build/include)
|
||||
|
||||
#include <iostream>
|
||||
#include <memory>
|
||||
|
||||
#include "gdal.h"
|
||||
|
||||
namespace gdal::sandbox::tests::parser {
|
||||
namespace gdal::sandbox::parser {
|
||||
|
||||
namespace {
|
||||
|
||||
|
|
@ -66,7 +65,6 @@ RasterDataset GetRasterBandsFromFile(const std::string& filename) {
|
|||
int data_type = static_cast<int>(GDALGetRasterDataType(band));
|
||||
int color_interp = static_cast<int>(GDALGetRasterColorInterpretation(band));
|
||||
|
||||
// Use std::variant or void* and reinterpet casts for the runtime template deduction
|
||||
std::vector<int> band_raster_data;
|
||||
band_raster_data.resize(width * height);
|
||||
|
||||
|
|
@ -88,6 +86,8 @@ RasterDataset GetRasterBandsFromFile(const std::string& filename) {
|
|||
|
||||
result_dataset.bands = std::move(bands_data);
|
||||
|
||||
GDALClose(dataset);
|
||||
|
||||
return result_dataset;
|
||||
}
|
||||
|
||||
|
|
@ -105,4 +105,4 @@ bool operator==(const RasterDataset& lhs, const RasterDataset& rhs) {
|
|||
&& lhs.geo_transform == rhs.geo_transform;
|
||||
}
|
||||
|
||||
} // namespace gdal::sandbox::tests:parser
|
||||
} // namespace gdal::sandbox::parser
|
||||
|
|
|
|||
|
|
@ -12,14 +12,14 @@
|
|||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef GET_RASTER_DATA_H_
|
||||
#define GET_RASTER_DATA_H_
|
||||
#ifndef RASTER_TO_GTIFF_GET_RASTER_DATA_H_
|
||||
#define RASTER_TO_GTIFF_GET_RASTER_DATA_H_
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
namespace gdal::sandbox::tests::parser {
|
||||
namespace gdal::sandbox::parser {
|
||||
|
||||
struct RasterBandData {
|
||||
int width;
|
||||
|
|
@ -42,6 +42,6 @@ RasterDataset GetRasterBandsFromFile(const std::string& filename);
|
|||
bool operator==(const RasterBandData& lhs, const RasterBandData& rhs);
|
||||
bool operator==(const RasterDataset& lhs, const RasterDataset& rhs);
|
||||
|
||||
} // namespace gdal::sandbox::tests::parser
|
||||
} // namespace gdal::sandbox::parser
|
||||
|
||||
#endif // GET_RASTER_DATA_H_
|
||||
#endif // RASTER_TO_GTIFF_GET_RASTER_DATA_H_
|
||||
|
|
|
|||
126
oss-internship-2020/gdal/raster_to_gtiff/gtiff_converter.cc
Normal file
126
oss-internship-2020/gdal/raster_to_gtiff/gtiff_converter.cc
Normal file
|
|
@ -0,0 +1,126 @@
|
|||
// Copyright 2020 Google LLC
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "gtiff_converter.h" // NOLINT(build/include)
|
||||
|
||||
namespace gdal::sandbox {
|
||||
|
||||
namespace {
|
||||
|
||||
inline constexpr absl::string_view kDriverName = "GTiff";
|
||||
|
||||
} // namespace
|
||||
|
||||
RasterToGTiffProcessor::RasterToGTiffProcessor(std::string out_file_full_path,
|
||||
std::string out_file_folder,
|
||||
std::string proj_db_path,
|
||||
parser::RasterDataset data,
|
||||
int retry_count)
|
||||
: sapi::Transaction(std::make_unique<GdalSapiSandbox>(
|
||||
std::move(out_file_folder), std::move(proj_db_path))),
|
||||
out_file_full_path_(std::move(out_file_full_path)),
|
||||
data_(std::move(data))
|
||||
{
|
||||
set_retry_count(retry_count);
|
||||
SetTimeLimit(absl::InfiniteDuration());
|
||||
}
|
||||
|
||||
absl::Status RasterToGTiffProcessor::Main() {
|
||||
gdalApi api(sandbox());
|
||||
SAPI_RETURN_IF_ERROR(api.GDALAllRegister());
|
||||
|
||||
sapi::v::ConstCStr driver_name_ptr(kDriverName.data());
|
||||
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<GDALDriverH> driver,
|
||||
api.GDALGetDriverByName(driver_name_ptr.PtrBefore()));
|
||||
|
||||
TRANSACTION_FAIL_IF_NOT(driver.value() != nullptr,
|
||||
"Error getting GTiff driver");
|
||||
sapi::v::RemotePtr driver_ptr(driver.value());
|
||||
|
||||
sapi::v::ConstCStr out_file_full_path_ptr(out_file_full_path_.c_str());
|
||||
sapi::v::NullPtr create_options;
|
||||
|
||||
GDALDataType type = data_.bands.size() > 0 ?
|
||||
static_cast<GDALDataType>(data_.bands[0].data_type)
|
||||
: GDALDataType::GDT_Unknown;
|
||||
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<GDALDatasetH> dataset,
|
||||
api.GDALCreate(&driver_ptr, out_file_full_path_ptr.PtrBefore(),
|
||||
data_.width, data_.height, data_.bands.size(), type,
|
||||
&create_options));
|
||||
|
||||
TRANSACTION_FAIL_IF_NOT(dataset.value(),
|
||||
"Error creating dataset");
|
||||
sapi::v::RemotePtr dataset_ptr(dataset.value());
|
||||
|
||||
for (int i = 0; i < data_.bands.size(); ++i) {
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<GDALRasterBandH> band,
|
||||
api.GDALGetRasterBand(&dataset_ptr, i + 1));
|
||||
TRANSACTION_FAIL_IF_NOT(band.value() != nullptr,
|
||||
"Error getting band from dataset");
|
||||
sapi::v::RemotePtr band_ptr(band.value());
|
||||
|
||||
sapi::v::Array<int> data_array(data_.bands[i].data.data(),
|
||||
data_.bands[i].data.size());
|
||||
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<CPLErr> result,
|
||||
api.GDALRasterIO(&band_ptr, GF_Write, 0, 0,
|
||||
data_.bands[i].width, data_.bands[i].height, data_array.PtrBefore(),
|
||||
data_.bands[i].width, data_.bands[i].height, GDT_Int32, 0, 0));
|
||||
|
||||
TRANSACTION_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error writing band to dataset");
|
||||
|
||||
SAPI_ASSIGN_OR_RETURN(result, api.GDALSetRasterColorInterpretation(
|
||||
&band_ptr, static_cast<GDALColorInterp>(
|
||||
data_.bands[i].color_interp)));
|
||||
|
||||
TRANSACTION_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error setting color interpretation");
|
||||
|
||||
if (data_.bands[i].no_data_value.has_value()) {
|
||||
SAPI_ASSIGN_OR_RETURN(result,
|
||||
api.GDALSetRasterNoDataValue(&band_ptr,
|
||||
data_.bands[i].no_data_value.value()));
|
||||
|
||||
TRANSACTION_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error setting no data value for the band");
|
||||
}
|
||||
}
|
||||
|
||||
if (data_.wkt_projection.length() > 0) {
|
||||
sapi::v::ConstCStr wkt_projection_ptr(data_.wkt_projection.c_str());
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<CPLErr> result,
|
||||
api.GDALSetProjection(&dataset_ptr, wkt_projection_ptr.PtrBefore()));
|
||||
TRANSACTION_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error setting wkt projection");
|
||||
}
|
||||
|
||||
if (data_.geo_transform.size() > 0) {
|
||||
sapi::v::Array<double> geo_transform_ptr(data_.geo_transform.data(),
|
||||
data_.geo_transform.size());
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<CPLErr> result,
|
||||
api.GDALSetGeoTransform(&dataset_ptr, geo_transform_ptr.PtrBefore()));
|
||||
|
||||
TRANSACTION_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error setting geo transform");
|
||||
}
|
||||
|
||||
SAPI_RETURN_IF_ERROR(api.GDALClose(&dataset_ptr));
|
||||
|
||||
return absl::OkStatus();
|
||||
}
|
||||
|
||||
} // namespace gdal::sandbox
|
||||
44
oss-internship-2020/gdal/raster_to_gtiff/gtiff_converter.h
Normal file
44
oss-internship-2020/gdal/raster_to_gtiff/gtiff_converter.h
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
// Copyright 2020 Google LLC
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef RASTER_TO_GTIFF_GTIFF_CONVERTER_H_
|
||||
#define RASTER_TO_GTIFF_GTIFF_CONVERTER_H_
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "sandboxed_api/transaction.h"
|
||||
|
||||
#include "gdal_sandbox.h" // NOLINT(build/include)
|
||||
#include "get_raster_data.h" // NOLINT(build/include)
|
||||
|
||||
namespace gdal::sandbox {
|
||||
|
||||
class RasterToGTiffProcessor : public sapi::Transaction {
|
||||
public:
|
||||
RasterToGTiffProcessor(std::string out_file_full_path,
|
||||
std::string out_file_folder,
|
||||
std::string proj_db_path,
|
||||
parser::RasterDataset data,
|
||||
int retry_count = 0);
|
||||
|
||||
private:
|
||||
absl::Status Main() final;
|
||||
|
||||
const std::string out_file_full_path_;
|
||||
parser::RasterDataset data_;
|
||||
};
|
||||
|
||||
} // namespace gdal::sandbox
|
||||
|
||||
#endif // RASTER_TO_GTIFF_GTIFF_CONVERTER_H_
|
||||
|
|
@ -13,43 +13,17 @@
|
|||
// limitations under the License.
|
||||
|
||||
#include <cstdlib>
|
||||
#include <filesystem>
|
||||
#include <iostream>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "sandboxed_api/sandbox2/util/fileops.h"
|
||||
|
||||
#include "gdal_sandbox.h" // NOLINT(build/include)
|
||||
#include "get_raster_data.h" // NOLINT(build/include)
|
||||
#include "gtiff_converter.h" // NOLINT(build/include)
|
||||
#include "utils.h" // NOLINT(build/include)
|
||||
|
||||
namespace {
|
||||
|
||||
#define SAPI_FAIL_IF_NOT(x, y) \
|
||||
if (!(x)) { \
|
||||
return absl::FailedPreconditionError(y); \
|
||||
}
|
||||
|
||||
inline constexpr absl::string_view kDriverName = "GTiff";
|
||||
inline constexpr absl::string_view kProjDbEnvVariableName = "PROJ_PATH";
|
||||
inline constexpr absl::string_view kDefaultProjDbPath
|
||||
= "/usr/local/share/proj/proj.db";
|
||||
|
||||
std::optional<std::string> GetProjDbPath() {
|
||||
const char* proj_db_path_ptr = std::getenv(kProjDbEnvVariableName.data());
|
||||
|
||||
std::string proj_db_path = proj_db_path_ptr == nullptr ?
|
||||
std::string(kDefaultProjDbPath) : std::string(proj_db_path_ptr);
|
||||
|
||||
if (!std::filesystem::exists(proj_db_path)) {
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
return proj_db_path;
|
||||
}
|
||||
|
||||
absl::Status SaveToGTiff(gdal::sandbox::tests::parser::RasterDataset bands_data,
|
||||
absl::Status SaveToGTiff(gdal::sandbox::parser::RasterDataset bands_data,
|
||||
std::string out_file) {
|
||||
using namespace gdal::sandbox;
|
||||
|
||||
|
|
@ -60,97 +34,16 @@ absl::Status SaveToGTiff(gdal::sandbox::tests::parser::RasterDataset bands_data,
|
|||
return absl::FailedPreconditionError("Error getting output file directory");
|
||||
}
|
||||
|
||||
std::optional<std::string> proj_db_path = GetProjDbPath();
|
||||
SAPI_FAIL_IF_NOT(proj_db_path != std::nullopt,
|
||||
"Specified proj.db does not exist");
|
||||
std::optional<std::string> proj_db_path = utils::FindProjDbPath();
|
||||
|
||||
GdalSapiSandbox sandbox(output_data_folder, std::move(proj_db_path.value()));
|
||||
SAPI_RETURN_IF_ERROR(sandbox.Init());
|
||||
gdalApi api(&sandbox);
|
||||
SAPI_RETURN_IF_ERROR(api.GDALAllRegister());
|
||||
|
||||
sapi::v::ConstCStr driver_name_ptr(kDriverName.data());
|
||||
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<GDALDriverH> driver,
|
||||
api.GDALGetDriverByName(driver_name_ptr.PtrBefore()));
|
||||
|
||||
SAPI_FAIL_IF_NOT(driver.value() != nullptr,
|
||||
"Error getting GTiff driver");
|
||||
sapi::v::RemotePtr driver_ptr(driver.value());
|
||||
|
||||
sapi::v::ConstCStr out_file_full_path_ptr(out_file.data());
|
||||
sapi::v::NullPtr create_options;
|
||||
|
||||
GDALDataType type = bands_data.bands.size() > 0 ?
|
||||
static_cast<GDALDataType>(bands_data.bands[0].data_type)
|
||||
: GDALDataType::GDT_Unknown;
|
||||
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<GDALDatasetH> dataset,
|
||||
api.GDALCreate(&driver_ptr, out_file_full_path_ptr.PtrBefore(),
|
||||
bands_data.width, bands_data.height, bands_data.bands.size(), type,
|
||||
&create_options));
|
||||
|
||||
SAPI_FAIL_IF_NOT(dataset.value(),
|
||||
"Error creating dataset");
|
||||
sapi::v::RemotePtr dataset_ptr(dataset.value());
|
||||
|
||||
for (int i = 0; i < bands_data.bands.size(); ++i) {
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<GDALRasterBandH> band,
|
||||
api.GDALGetRasterBand(&dataset_ptr, i + 1));
|
||||
SAPI_FAIL_IF_NOT(band.value() != nullptr,
|
||||
"Error getting band from dataset");
|
||||
sapi::v::RemotePtr band_ptr(band.value());
|
||||
|
||||
sapi::v::Array<int> data_array(bands_data.bands[i].data.data(),
|
||||
bands_data.bands[i].data.size());
|
||||
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<CPLErr> result,
|
||||
api.GDALRasterIO(&band_ptr, GF_Write, 0, 0,
|
||||
bands_data.bands[i].width, bands_data.bands[i].height,
|
||||
data_array.PtrBefore(), bands_data.bands[i].width,
|
||||
bands_data.bands[i].height, GDALDataType::GDT_Int32, 0, 0));
|
||||
|
||||
SAPI_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error writing band to dataset");
|
||||
|
||||
SAPI_ASSIGN_OR_RETURN(result, api.GDALSetRasterColorInterpretation(
|
||||
&band_ptr, static_cast<GDALColorInterp>(
|
||||
bands_data.bands[i].color_interp)));
|
||||
|
||||
SAPI_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error setting color interpretation");
|
||||
|
||||
if (bands_data.bands[i].no_data_value.has_value()) {
|
||||
SAPI_ASSIGN_OR_RETURN(result,
|
||||
api.GDALSetRasterNoDataValue(&band_ptr,
|
||||
bands_data.bands[i].no_data_value.value()));
|
||||
|
||||
SAPI_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error setting no data value for the band");
|
||||
}
|
||||
if (proj_db_path == std::nullopt) {
|
||||
return absl::FailedPreconditionError("Specified proj.db does not exist");
|
||||
}
|
||||
|
||||
if (bands_data.wkt_projection.length() > 0) {
|
||||
sapi::v::ConstCStr wkt_projection_ptr(bands_data.wkt_projection.data());
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<CPLErr> result,
|
||||
api.GDALSetProjection(&dataset_ptr, wkt_projection_ptr.PtrBefore()));
|
||||
SAPI_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error setting wkt projection");
|
||||
}
|
||||
RasterToGTiffProcessor processor(out_file, output_data_folder,
|
||||
proj_db_path.value(), bands_data);
|
||||
|
||||
if (bands_data.geo_transform.size() > 0) {
|
||||
sapi::v::Array<double> geo_transform_ptr(bands_data.geo_transform.data(),
|
||||
bands_data.geo_transform.size());
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<CPLErr> result,
|
||||
api.GDALSetGeoTransform(&dataset_ptr, geo_transform_ptr.PtrBefore()));
|
||||
|
||||
SAPI_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error setting geo transform");
|
||||
}
|
||||
|
||||
SAPI_RETURN_IF_ERROR(api.GDALClose(&dataset_ptr));
|
||||
|
||||
return absl::OkStatus();
|
||||
return processor.Run();
|
||||
}
|
||||
|
||||
void Usage() {
|
||||
|
|
@ -165,8 +58,7 @@ void Usage() {
|
|||
int main(int argc, char** argv) {
|
||||
using namespace gdal::sandbox;
|
||||
|
||||
if (argc < 3
|
||||
|| !std::filesystem::path(std::string(argv[2])).is_absolute()) {
|
||||
if (argc < 3 || !gdal::sandbox::utils::IsAbsolute(argv[2])) {
|
||||
Usage();
|
||||
return EXIT_FAILURE;
|
||||
}
|
||||
|
|
@ -174,11 +66,9 @@ int main(int argc, char** argv) {
|
|||
std::string input_data_path = std::string(argv[1]);
|
||||
std::string output_data_path = std::string(argv[2]);
|
||||
|
||||
tests::parser::RasterDataset bands_data
|
||||
= tests::parser::GetRasterBandsFromFile(std::move(input_data_path));
|
||||
|
||||
if (absl::Status status =
|
||||
SaveToGTiff(std::move(bands_data), std::move(output_data_path));
|
||||
SaveToGTiff(parser::GetRasterBandsFromFile(std::move(input_data_path)),
|
||||
std::move(output_data_path));
|
||||
!status.ok()) {
|
||||
std::cerr << status.ToString() << std::endl;
|
||||
return EXIT_FAILURE;
|
||||
|
|
|
|||
|
|
@ -12,193 +12,26 @@
|
|||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include <filesystem>
|
||||
#include <iostream>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
|
||||
#include "gtest/gtest.h"
|
||||
#include "sandboxed_api/sandbox2/util/fileops.h"
|
||||
#include "sandboxed_api/sandbox2/util/temp_file.h"
|
||||
#include "sandboxed_api/transaction.h"
|
||||
|
||||
#include "gdal_sandbox.h" // NOLINT(build/include)
|
||||
#include "get_raster_data.h" // NOLINT(build/include)
|
||||
#include "gtiff_converter.h" // NOLINT(build/include)
|
||||
#include "utils.h" // NOLINT(build/include)
|
||||
|
||||
namespace gdal::sandbox::tests {
|
||||
|
||||
namespace {
|
||||
|
||||
inline constexpr absl::string_view kDriverName = "GTiff";
|
||||
inline constexpr absl::string_view kTempFilePrefix = "temp_data";
|
||||
inline constexpr absl::string_view kProjDbEnvVariableName = "PROJ_PATH";
|
||||
inline constexpr absl::string_view kDefaultProjDbPath
|
||||
= "/usr/local/share/proj/proj.db";
|
||||
inline constexpr absl::string_view kFirstTestDataPath =
|
||||
"../testdata/cea.tif";
|
||||
inline constexpr absl::string_view kSecondTestDataPath =
|
||||
"../testdata/SP27GTIF.tif";
|
||||
|
||||
std::optional<std::string> GetProjDbPath() {
|
||||
const char* proj_db_path_ptr = std::getenv(kProjDbEnvVariableName.data());
|
||||
|
||||
std::string proj_db_path = proj_db_path_ptr == nullptr ?
|
||||
std::string(kDefaultProjDbPath) : std::string(proj_db_path_ptr);
|
||||
|
||||
if (!std::filesystem::exists(proj_db_path)) {
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
return proj_db_path;
|
||||
}
|
||||
|
||||
// RAII wrapper that creates temporary file and automatically unlinks it
|
||||
class TempFile {
|
||||
public:
|
||||
explicit TempFile(absl::string_view prefix)
|
||||
{
|
||||
auto file_data = sandbox2::CreateNamedTempFile(prefix);
|
||||
|
||||
if (file_data.ok()) {
|
||||
file_data_ = std::move(file_data.value());
|
||||
}
|
||||
}
|
||||
|
||||
~TempFile() {
|
||||
if (file_data_.has_value()) {
|
||||
unlink(file_data_.value().first.c_str());
|
||||
}
|
||||
}
|
||||
|
||||
bool HasValue() const {
|
||||
return file_data_.has_value();
|
||||
}
|
||||
|
||||
int GetFd() const {
|
||||
return file_data_.value().second;
|
||||
}
|
||||
|
||||
std::string GetPath() const {
|
||||
return file_data_.value().first;
|
||||
}
|
||||
|
||||
private:
|
||||
std::optional<std::pair<std::string, int>> file_data_ = std::nullopt;
|
||||
};
|
||||
|
||||
// Wrapper around raster to GTiff workflow
|
||||
class RasterToGTiffProcessor : public sapi::Transaction {
|
||||
public:
|
||||
explicit RasterToGTiffProcessor(std::string out_filename,
|
||||
std::string out_path,
|
||||
std::string proj_db_path,
|
||||
parser::RasterDataset data,
|
||||
int retry_count = 0)
|
||||
: sapi::Transaction(std::make_unique<GdalSapiSandbox>(out_path,
|
||||
std::move(proj_db_path))),
|
||||
out_filename_(std::move(out_filename)),
|
||||
out_path_(std::move(out_path)),
|
||||
data_(std::move(data))
|
||||
{
|
||||
set_retry_count(retry_count);
|
||||
SetTimeLimit(absl::InfiniteDuration());
|
||||
}
|
||||
|
||||
private:
|
||||
absl::Status Main() final;
|
||||
|
||||
const std::string out_filename_;
|
||||
const std::string out_path_;
|
||||
parser::RasterDataset data_;
|
||||
};
|
||||
|
||||
absl::Status RasterToGTiffProcessor::Main() {
|
||||
gdalApi api(sandbox());
|
||||
SAPI_RETURN_IF_ERROR(api.GDALAllRegister());
|
||||
|
||||
sapi::v::ConstCStr driver_name_ptr(kDriverName.data());
|
||||
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<GDALDriverH> driver,
|
||||
api.GDALGetDriverByName(driver_name_ptr.PtrBefore()));
|
||||
|
||||
TRANSACTION_FAIL_IF_NOT(driver.value() != nullptr,
|
||||
"Error getting GTiff driver");
|
||||
sapi::v::RemotePtr driver_ptr(driver.value());
|
||||
|
||||
std::string out_file_full_path = absl::StrCat(out_path_, "/", out_filename_);
|
||||
sapi::v::ConstCStr out_file_full_path_ptr(out_file_full_path.data());
|
||||
sapi::v::NullPtr create_options;
|
||||
|
||||
GDALDataType type = data_.bands.size() > 0 ?
|
||||
static_cast<GDALDataType>(data_.bands[0].data_type)
|
||||
: GDALDataType::GDT_Unknown;
|
||||
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<GDALDatasetH> dataset,
|
||||
api.GDALCreate(&driver_ptr, out_file_full_path_ptr.PtrBefore(),
|
||||
data_.width, data_.height, data_.bands.size(), type,
|
||||
&create_options));
|
||||
|
||||
TRANSACTION_FAIL_IF_NOT(dataset.value(),
|
||||
"Error creating dataset");
|
||||
sapi::v::RemotePtr dataset_ptr(dataset.value());
|
||||
|
||||
for (int i = 0; i < data_.bands.size(); ++i) {
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<GDALRasterBandH> band,
|
||||
api.GDALGetRasterBand(&dataset_ptr, i + 1));
|
||||
TRANSACTION_FAIL_IF_NOT(band.value() != nullptr,
|
||||
"Error getting band from dataset");
|
||||
sapi::v::RemotePtr band_ptr(band.value());
|
||||
|
||||
sapi::v::Array<int> data_array(data_.bands[i].data.data(),
|
||||
data_.bands[i].data.size());
|
||||
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<CPLErr> result,
|
||||
api.GDALRasterIO(&band_ptr, GF_Write, 0, 0,
|
||||
data_.bands[i].width, data_.bands[i].height, data_array.PtrBefore(),
|
||||
data_.bands[i].width, data_.bands[i].height, GDT_Int32, 0, 0));
|
||||
|
||||
TRANSACTION_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error writing band to dataset");
|
||||
|
||||
SAPI_ASSIGN_OR_RETURN(result, api.GDALSetRasterColorInterpretation(
|
||||
&band_ptr, static_cast<GDALColorInterp>(
|
||||
data_.bands[i].color_interp)));
|
||||
|
||||
TRANSACTION_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error setting color interpretation");
|
||||
|
||||
if (data_.bands[i].no_data_value.has_value()) {
|
||||
SAPI_ASSIGN_OR_RETURN(result,
|
||||
api.GDALSetRasterNoDataValue(&band_ptr,
|
||||
data_.bands[i].no_data_value.value()));
|
||||
|
||||
TRANSACTION_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error setting no data value for the band");
|
||||
}
|
||||
}
|
||||
|
||||
if (data_.wkt_projection.length() > 0) {
|
||||
sapi::v::ConstCStr wkt_projection_ptr(data_.wkt_projection.data());
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<CPLErr> result,
|
||||
api.GDALSetProjection(&dataset_ptr, wkt_projection_ptr.PtrBefore()));
|
||||
TRANSACTION_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error setting wkt projection");
|
||||
}
|
||||
|
||||
if (data_.geo_transform.size() > 0) {
|
||||
sapi::v::Array<double> geo_transform_ptr(data_.geo_transform.data(),
|
||||
data_.geo_transform.size());
|
||||
SAPI_ASSIGN_OR_RETURN(sapi::StatusOr<CPLErr> result,
|
||||
api.GDALSetGeoTransform(&dataset_ptr, geo_transform_ptr.PtrBefore()));
|
||||
|
||||
TRANSACTION_FAIL_IF_NOT(result.value() == CPLErr::CE_None,
|
||||
"Error setting geo transform");
|
||||
}
|
||||
|
||||
SAPI_RETURN_IF_ERROR(api.GDALClose(&dataset_ptr));
|
||||
|
||||
return absl::OkStatus();
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
class TestGTiffProcessor : public testing::TestWithParam<absl::string_view> {
|
||||
|
|
@ -208,7 +41,7 @@ class TestGTiffProcessor : public testing::TestWithParam<absl::string_view> {
|
|||
{}
|
||||
|
||||
protected:
|
||||
const TempFile tempfile_;
|
||||
const utils::TempFile tempfile_;
|
||||
};
|
||||
|
||||
TEST_P(TestGTiffProcessor, TestProcessorOnGTiffData) {
|
||||
|
|
@ -220,11 +53,12 @@ TEST_P(TestGTiffProcessor, TestProcessorOnGTiffData) {
|
|||
parser::RasterDataset original_bands_data =
|
||||
parser::GetRasterBandsFromFile(filename);
|
||||
|
||||
std::optional<std::string> proj_db_path = GetProjDbPath();
|
||||
std::optional<std::string> proj_db_path = utils::FindProjDbPath();
|
||||
ASSERT_TRUE(proj_db_path != std::nullopt)
|
||||
<< "Specified proj.db does not exist";
|
||||
|
||||
RasterToGTiffProcessor processor(tempfile_.GetPath(),
|
||||
RasterToGTiffProcessor processor(absl::StrCat(
|
||||
sandbox2::file_util::fileops::GetCWD(), "/", tempfile_.GetPath()),
|
||||
sandbox2::file_util::fileops::GetCWD(), std::move(proj_db_path.value()),
|
||||
original_bands_data);
|
||||
|
||||
|
|
|
|||
75
oss-internship-2020/gdal/raster_to_gtiff/utils.cc
Normal file
75
oss-internship-2020/gdal/raster_to_gtiff/utils.cc
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
// Copyright 2020 Google LLC
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "utils.h" // NOLINT(build/include)
|
||||
|
||||
#include <unistd.h>
|
||||
|
||||
#include "sandboxed_api/sandbox2/util/fileops.h"
|
||||
#include "sandboxed_api/sandbox2/util/temp_file.h"
|
||||
|
||||
namespace gdal::sandbox::utils {
|
||||
|
||||
namespace {
|
||||
|
||||
inline constexpr absl::string_view kProjDbEnvVariableName = "PROJ_PATH";
|
||||
inline constexpr absl::string_view kDefaultProjDbPath
|
||||
= "/usr/local/share/proj/proj.db";
|
||||
|
||||
} // namespace
|
||||
|
||||
TempFile::TempFile(absl::string_view prefix) {
|
||||
auto file_data = sandbox2::CreateNamedTempFile(prefix);
|
||||
|
||||
if (file_data.ok()) {
|
||||
file_data_ = std::move(file_data.value());
|
||||
}
|
||||
}
|
||||
|
||||
TempFile::~TempFile() {
|
||||
if (HasValue()) {
|
||||
unlink(file_data_.value().first.c_str());
|
||||
}
|
||||
}
|
||||
|
||||
bool TempFile::HasValue() const {
|
||||
return file_data_.has_value();
|
||||
}
|
||||
|
||||
int TempFile::GetFd() const {
|
||||
return file_data_.value().second;
|
||||
}
|
||||
|
||||
std::string TempFile::GetPath() const {
|
||||
return file_data_.value().first;
|
||||
}
|
||||
|
||||
std::optional<std::string> FindProjDbPath() {
|
||||
const char* proj_db_path_ptr = std::getenv(kProjDbEnvVariableName.data());
|
||||
|
||||
std::string proj_db_path = proj_db_path_ptr == nullptr ?
|
||||
std::string(kDefaultProjDbPath) : std::string(proj_db_path_ptr);
|
||||
|
||||
if (!sandbox2::file_util::fileops::Exists(proj_db_path, false)) {
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
return proj_db_path;
|
||||
}
|
||||
|
||||
bool IsAbsolute(absl::string_view path) {
|
||||
return path.length() > 0 && path.at(0) == '/';
|
||||
}
|
||||
|
||||
} // namespace gdal::sandbox::utils
|
||||
49
oss-internship-2020/gdal/raster_to_gtiff/utils.h
Normal file
49
oss-internship-2020/gdal/raster_to_gtiff/utils.h
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
// Copyright 2020 Google LLC
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef RASTER_TO_GTIFF_UTILS_H_
|
||||
#define RASTER_TO_GTIFF_UTILS_H
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
|
||||
#include "absl/strings/string_view.h"
|
||||
|
||||
namespace gdal::sandbox::utils {
|
||||
|
||||
// RAII wrapper that creates temporary file and automatically unlinks it
|
||||
class TempFile {
|
||||
public:
|
||||
explicit TempFile(absl::string_view prefix);
|
||||
TempFile(const TempFile&) = delete;
|
||||
TempFile& operator=(const TempFile&) = delete;
|
||||
TempFile(TempFile&&) = delete;
|
||||
TempFile& operator=(TempFile&&) = delete;
|
||||
~TempFile();
|
||||
|
||||
bool HasValue() const;
|
||||
int GetFd() const;
|
||||
std::string GetPath() const;
|
||||
|
||||
private:
|
||||
std::optional<std::pair<std::string, int>> file_data_ = std::nullopt;
|
||||
};
|
||||
|
||||
// Helper function to retrieve potential proj.db path from environment variable
|
||||
std::optional<std::string> FindProjDbPath();
|
||||
bool IsAbsolute(absl::string_view path);
|
||||
|
||||
} // namespace gdal::sandbox::utils
|
||||
|
||||
#endif // RASTER_TO_GTIFF_UTILS_H_
|
||||
Loading…
Reference in New Issue
Block a user