Integration of TileJSON with MicroJSON

Purpose

This specification outlines how to use TileJSON to integrate tiled MicroJSON data, both in json form as well as binary form. It provides examples of how TileJSON can be used to specify the tiling scheme and zoom levels for MicroJSON data and its binary equievalent. It is based on the TileJSON 3.0.0 specification, but extends it by recommending additional properties to enable integration of MicroJSON data and fit purposes of microscopy imaging. The recommendations provided here are not intrinsic to the original TileJSON specification but have been tailored to suit the needs of microscopy metadata annotation and integration with MicroJSON. However, all suggestions are designed to maintain compatibility with the original TileJSON specification.

Background of TileJSON

TileJSON is a widely-used format in mapping applications for specifying tilesets. Developed to streamline the integration of different map layers, TileJSON is essential for ensuring consistency across mapping platforms. It describes tilesets through a JSON object, detailing properties like tile URLs, zoom levels, and spatial coverage.

TileJSON for MicroJSON Object Structure

tilejson: Specifies the version of the TileJSON spec being used. Required for all TileJSON objects.
name: The name of the tileset. Optional but recommended.
description: Provide a brief description of the tileset. Optional but recommended.
version: The version of the tileset. Optional but recommended.
attribution: A link to the data source or other attribution information, e.g. organisational origin. Optional but recommended.
tiles: Required. The URL pattern for accessing the vector tiles. The urlbase/{zlvl}/{t}/{c}/{z}/{x}/{y} is the recommended default naming pattern for the tiles, in this order, where urlbase is the base URL (e.g. http://example.com/tiles), {zlvl} is the zoom level, {t} is the tileset timestamp, {c} is the channel, {z} is the z coordinate, and {x} and {y} are the x and y coordinates, respectively. If not using a timestamp, channel, or z coordinate, these can be omitted. The zoom level should always be first.
minzoom and maxzoom: Defines the range of zoom levels for which the tiles are available.
bounds: Optional. Specifies the geometrical bounds included in the tileset. Specified as an array of minimum four numbers in the order [minX, minY, maxX, maxY], but may include up to a further six numbers for a total of ten, [minT, minC, minZ, minX, minY, maxT, maxC, maxZ, maxX, maxY], where minT is the minimum tileset timestamp, minC is the minimum channel, minZ is the minimum z coordinate, minX and minY are the minimum x and y coordinates, maxT is the maximum tileset timestamp, maxC is the maximum channel, maxZ is the maximum z coordinate, and maxX and maxY are the maximum x and y coordinates.
center: Optional. Indicates the center and suggested default view of the tileset. Minimum of three numbers in the order [x, y, zoom], but may include up to a further three numbers for a total of six, [t,c,z,x,y,zoom], where t is the tileset timestamp, c is the channel, z is the z coordinate, x and y are the x and y coordinates, and zoom is the zoom level. Zoom level should be last.
vector_layers: Required. Describes each layer within the vector tiles, and has the following structure:
id: Required. A unique identifier for the layer. Required for each layer.
fields: Required. A list of fields (attributes) and their data types. For MicroJSON, this can either be an empty list, or a simple datatype indicator, that is either of String, Number, or Bool. Complex data types, such as arrays or objects are not allowed. Required for each layer.
fieldranges: Optional. A dictionary of field names and their ranges. For example, {"label": [0,100], "channel": [0,10]}. Optional.
fieldenums: Optional. A dictionary of field names and their possible values. For example, {"plate": ["A1", "A2", "B1", "B2"], "image": ["image1.tif", "image2.tif", "image3.tif"]}. Optional.
fielddescriptions: Optional. A dictionary of field names and their descriptions. For example, {"plate": "Well plate identifier", "image": "Image filename", "label": "Label identifier", "channel": "Channel identifier"}. Optional.
description: Optional. A brief description of the layer.
minzoom and maxzoom: Optional. The range of zoom levels at which the layer is visible.
fillzoom: Optional. An integer specifying the zoom level from which to generate overzoomed tiles.
legend: Optional. Contains a legend to be displayed with the tileset.
multiscale: Optional. A multiscale object as defined in the section Multiscale Object. If this property is not present, the default coordinate system is assumed to be the same as the image coordinate system, using cartesian coordinates and pixels as units.
scale_factor: Optional. A float specifying the scale factor for the tileset. If not present, the default value of 2 is assumed. Unless the use case requires a different scale factor, it is highly recommended to use the default value, as it is widely supported and assumed by many viewers.

The following fields of TileJSON may be used if the use case requires it, and are included here for completeness:

scheme: The tiling scheme of the tileset.
grids: The URL pattern for accessing grid data.
data: Optional. The URL pattern for accessing data. Used for GeoJSON originally, which in this specification is replaced by MicroJSON and used in the tiles field.
template: Optional. Contains a mustache template to be used to format data from grids for interaction.

Multiscale Object

A multiscale object represents the choice of axes (2-5D) and potentially their transformations that should be applied to the numerical data in order to arrive to the actual size of the object described. If the field is present, it MUST have the following properties:

"axes": Representing the choice of axes as an array of Axis objects.

It may contain either of, but NOT both of the following properties:

"coordinateTransformations": Representing the set of coordinate transformations that should be applied to the numerical data in order to arrive to the actual size of the object described. It MUST be an array of objects, each object representing a coordinate transformation. Each object MUST have properties as follows:
"type": Representing the type of the coordinate transformation. Currently supported types are "identity", "scale", and "translate". If the type is "scale", the object MUST have the property "scale", representing the scaling factor. It MUST be an array of numbers, with the number of elements equal to the number of axes in the coordinate system. If the type is "translate", the object MUST have the property "translate", representing the translation vector. It MUST be an array of numbers, with the number of elements equal to the number of axes in the coordinate system. If the type is "identity", the object MUST NOT have any other properties.
"transformationMatrix": Representing the transformation matrix from the coordinate system of the image to the coordinate system of the MicroJSON object. It MUST be an array of arrays of numbers, with the number of rows equal to the number of axes in the coordinate system, and the number of columns equal to the number of axes in the image coordinate system. The transformation matrix MUST be invertible.

Axis Object

Together with the other axes in the axes array, an axis object represents the coordinate system of the MicroJSON object (2D-5D) It MUST have the following properties:

"name": Representing the name of the axis. It MUST be a string. It may contain the following properties:
"unit": Representing the units of the corresponding axis of the geometries in the MicroJSON object. It MUST be an array with the elements having any of the following values: [“angstrom", "attometer", "centimeter", "decimeter", "exameter", "femtometer", "foot", "gigameter", "hectometer", "inch", "kilometer", "megameter", "meter", "micrometer", "mile", "millimeter", "nanometer", "parsec", "petameter", "picometer", "terameter", "yard", "yoctometer", "yottameter", "zeptometer", "zettameter“]
"description": A string describing the axis.

Pydantic Model for TileJSON for MicroJSON

TileJSON

Bases: RootModel

The root object of a TileJSON file.

Source code in src/microjson/tilemodel.py

class TileJSON(RootModel):
    """ The root object of a TileJSON file."""
    root: TileModel

TileModel

Bases: BaseModel

A TileJSON object.

Parameters:

Name	Type	Description	Default
`tilejson`	`str`	The TileJSON version.	required
`tiles`	`List[Union[Path, AnyUrl]]`	The list of tile URLs.	required
`name`	`Optional[str]`	The name of the tileset.	required
`description`	`Optional[str]`	The description of the tileset.	required
`version`	`Optional[str]`	The version of the tileset.	required
`attribution`	`Optional[str]`	The attribution of the tileset.	required
`template`	`Optional[str]`	The template of the tileset.	required
`legend`	`Optional[str]`	The legend of the tileset.	required
`scheme`	`Optional[str]`	The scheme of the tileset.	required
`grids`	`Optional[Union[Path, AnyUrl]]`	The grids of the tileset.	required
`data`	`Optional[Union[Path, AnyUrl]]`	The data of the tileset.	required
`minzoom`	`Optional[int]`	The minimum zoom level of the tileset.	required
`maxzoom`	`Optional[int]`	The maximum zoom level of the tileset.	required
`bounds`	`Optional[conlist(float, min_length=4, max_length=10)]`	The bounds of the tileset.	required
`center`	`Optional[conlist(float, min_length=3, max_length=6)]`	The center of the tileset.	required
`fillzoom`	`Optional[int]`	The fill zoom level of the tileset.	required
`vector_layers`	`List[TileLayer]`	The vector layers of the tileset.	required

Source code in src/microjson/tilemodel.py

class TileModel(BaseModel):
    """ A TileJSON object.

    Args:
        tilejson (str): The TileJSON version.
        tiles (List[Union[Path, AnyUrl]]): The list of tile URLs.
        name (Optional[str]): The name of the tileset.
        description (Optional[str]): The description of the tileset.
        version (Optional[str]): The version of the tileset.
        attribution (Optional[str]): The attribution of the tileset.
        template (Optional[str]): The template of the tileset.
        legend (Optional[str]): The legend of the tileset.
        scheme (Optional[str]): The scheme of the tileset.
        grids (Optional[Union[Path, AnyUrl]]): The grids of the tileset.
        data (Optional[Union[Path, AnyUrl]]): The data of the tileset.
        minzoom (Optional[int]): The minimum zoom level of the tileset.
        maxzoom (Optional[int]): The maximum zoom level of the tileset.
        bounds (Optional[conlist(float, min_length=4, max_length=10)]):
            The bounds of the tileset.
        center (Optional[conlist(float, min_length=3, max_length=6)]):
            The center of the tileset.
        fillzoom (Optional[int]): The fill zoom level of the tileset.
        vector_layers (List[TileLayer]): The vector layers of the tileset.

    """

    tilejson: str
    tiles: List[Union[Path, AnyUrl]]
    name: Optional[str] = None
    description: Optional[str] = None
    version: Optional[str] = None
    attribution: Optional[str] = None
    template: Optional[str] = None
    legend: Optional[str] = None
    scheme: Optional[str] = None
    grids: Optional[Union[Path, AnyUrl]] = None
    data: Optional[Union[Path, AnyUrl]] = None
    minzoom: Optional[int] = 0
    maxzoom: Optional[int] = 22
    bounds: Optional[conlist(  # type: ignore
        float,
        min_length=4,
        max_length=10)] = None
    center: Optional[conlist(  # type: ignore
        float,
        min_length=3,
        max_length=6)] = None
    fillzoom: Optional[int] = None
    vector_layers: List[TileLayer]
    multiscale: Optional[Multiscale] = None
    scale_factor: Optional[float] = None

TileLayer

Bases: BaseModel

A vector layer in a TileJSON file.

Parameters:

Name	Type	Description	Default
`id`	`str`	The unique identifier for the layer.	required
`fields`	`Union[None, Dict[str, str]]`	The fields in the layer.	required
`minzoom`	`Optional[int]`	The minimum zoom level for the layer.	required
`maxzoom`	`Optional[int]`	The maximum zoom level for the layer.	required
`description`	`Optional[str]`	A description of the layer.	required
`fieldranges`	`Optional[Dict[str, List[Union[int, float, str]]]]`	The ranges of the fields.	required
`fieldenums`	`Optional[Dict[str, List[str]]]`	The enums of the fields.	required
`fielddescriptions`	`Optional[Dict[str, str]]`	The descriptions of the fields.	required

Source code in src/microjson/tilemodel.py

class TileLayer(BaseModel):
    """ A vector layer in a TileJSON file.

    Args:
        id (str): The unique identifier for the layer.
        fields (Union[None, Dict[str, str]]): The fields in the layer.
        minzoom (Optional[int]): The minimum zoom level for the layer.
        maxzoom (Optional[int]): The maximum zoom level for the layer.
        description (Optional[str]): A description of the layer.
        fieldranges (Optional[Dict[str, List[Union[int, float, str]]]]):
            The ranges of the fields.
        fieldenums (Optional[Dict[str, List[str]]]):
            The enums of the fields.
        fielddescriptions (Optional[Dict[str, str]]):
            The descriptions of the fields.
    """
    id: str
    fields: Union[None, Dict[str, str]] = None
    minzoom: Optional[int] = 0
    maxzoom: Optional[int] = 22
    description: Optional[str] = None
    fieldranges: Optional[Dict[str, List[Union[int, float, str]]]] = None
    fieldenums: Optional[Dict[str, List[str]]] = None
    fielddescriptions: Optional[Dict[str, str]] = None

General tiling requirements

This specification is designed to be compatible with the Vector Tile Specification and the TileJSON 3.0.0 specification. The Vector Tile Specification specifically requires vector tiles to be agnostic of the global coordinate system, and thus followingly each tile has a relative coordinate system, which instead is defined in the TileJSON. Our ambitions in general are to follow the same principles.

One difference is that we here recommend that the file ending for binary tiles is .pbf instead of .mvt to avoid confusion with the Mapbox Vector Tile format. The binary tiles should be encoded in the Protobuf format as defined in the Vector Tile Specification.

MicroJSON2vt

The MicroJSON2vt module is a helper module that can be used to convert MicroJSON objects to vector tiles. It is designed to be used in conjunction with the TileJSON for MicroJSON specification, and can be used to generate vector tiles from MicroJSON objects. The module is designed to be compatible with the Vector Tile Specification, and can be used to generate vector tiles in the intermediate vector tile JSON-format, which then, using vt2pbf may be transformed into protobuf. The module is included in the microjson package, and its wrapper function can be imported using the following code:

from microjson import microjson2vt

The module:

MicroJsonVt class, which is the main class for generating vector tiles from MicroJSON data

Source code in src/microjson/microjson2vt/microjson2vt.py

class MicroJsonVt:
    """
    MicroJsonVt class, which is the main class for generating vector tiles
    from MicroJSON data
    """
    def __init__(self, data, options, log_level=logging.INFO):
        """
        Constructor for MicroJsonVt class

        Args:
            data (dict): The data to be converted to vector tiles
            options (dict): The options to be used for generating vector tiles
            log_level (int): The logging level to be used
        """
        logging.basicConfig(
            level=log_level, format='%(asctime)s %(levelname)s %(message)s')
        options = self.options = extend(get_default_options(), options)

        # Validate and resolve tolerance_function
        tolerance_setting = options.get('tolerance_function')
        if isinstance(tolerance_setting, str):
            if tolerance_setting in AVAILABLE_TOLERANCE_FUNCTIONS:
                options['tolerance_function'] = AVAILABLE_TOLERANCE_FUNCTIONS[tolerance_setting]
            else:
                raise ValueError(
                    f"Invalid tolerance function key: '{tolerance_setting}'. "
                    f"Available keys: {list(AVAILABLE_TOLERANCE_FUNCTIONS.keys())}"
                )
        elif not callable(tolerance_setting):
            raise TypeError(
                "Option 'tolerance_function' must be a callable function or a valid string key."
            )
        # If it's already callable, we use it directly.

        logging.debug('preprocess data start')

        if options.get('maxZoom') < 0 or options.get('maxZoom') > 24:
            raise Exception('maxZoom should be in the 0-24 range')
        if options.get(
            'promoteId', None) is not None and options.get(
                'generateId', False):
            raise Exception(
                'promoteId and generateId cannot be used together.')

        # projects and adds simplification info
        # Create a new instance of a CartesianProjector

        features = convert(data, options)

        # Create a separate geometry for each zoom level
        for z in range(options.get('maxZoom') + 1):
            for feature in features:
                feature[f'geometry_z{z}'] = feature['geometry'].copy()

        tolerance_func = options['tolerance_function'] # Resolved above

        # Simplify features for each zoom level
        for z in range(options.get('maxZoom') + 1):
            # Calculate tolerance using the provided or default function
            tolerance = tolerance_func(z, options)
            for feature in features:
                geometry_key = f'geometry_z{z}'
                # check feature type only simplify Polygon
                if feature['type'] == 'Polygon':
                    for iring in range(len(feature[geometry_key])):
                        ring = feature[geometry_key][iring]
                        # Convert geom to list of [x, y] pairs
                        coords = [[ring[i], ring[i + 1]] for i in range(
                            0, len(ring), 3)]
                        scoords = simplify(coords, tolerance)
                        # Check that it has at least 4 pairs of coordinates
                        if len(scoords) < 4:
                            # If not, use the original coordinates
                            feature[geometry_key][iring] = ring
                        else:
                            # flatten the simplified coords
                            simplified_ring = []
                            for i in range(len(scoords)):
                                simplified_ring.append(scoords[i][0])
                                simplified_ring.append(scoords[i][1])
                                simplified_ring.append(0)
                            feature[geometry_key][iring] = simplified_ring

        # tiles and tile_coords are part of the public API
        self.tiles = {}
        self.tile_coords = []

        self.stats = {}
        self.total = 0

        # wraps features (ie extreme west and extreme east)
        # features = wrap(features, options)

        # start slicing from the top tile down
        if len(features) > 0:
            self.split_tile(features, 0, 0, 0)

    # splits features from a parent tile to sub-tiles.
    # z, x, and y are the coordinates of the parent tile
    # cz, cx, and cy are the coordinates of the target tile
    #
    # If no target tile is specified, splitting stops when we reach the maximum
    # zoom or the number of points is low as specified in the options.

    def split_tile(self, features, z, x, y, cz=None, cx=None, cy=None):
        """
        Splits features from a parent tile to sub-tiles.

        Args:
            features (list): The features to be split
            z (int): The zoom level of the parent tile
            x (int): The x coordinate of the parent tile
            y (int): The y coordinate of the parent tile
            cz (int): The zoom level of the target tile
            cx (int): The x coordinate of the target tile
            cy (int): The y coordinate of the target tile
        """
        stack = [features, z, x, y]
        options = self.options
        # avoid recursion by using a processing queue
        while len(stack) > 0:
            y = stack.pop()
            x = stack.pop()
            z = stack.pop()
            features = stack.pop()

            z2 = 1 << z
            id_ = to_Id(z, x, y)
            tile = self.tiles.get(id_, None)

            if tile is None:
                # Use simplified geometries for this zoom level

                simplified_features = [
                    {
                        **feature,
                        "geometry": feature[f'geometry_z{z}']
                    }
                    for feature in features
                ]

                self.tiles[id_] = create_tile(features, z, x, y, options)
                tile = self.tiles[id_]
                self.tile_coords.append({'z': z, 'x': x, 'y': y})

                key = f'z{z}'
                self.stats[key] = self.stats.get(key, 0) + 1
                self.total += 1

                self.tiles[id_] = create_tile(
                    simplified_features, z, x, y, options)
                tile = self.tiles[id_]
                self.tile_coords.append({'z': z, 'x': x, 'y': y})

                self.stats[f'z{z}'] = self.stats.get(f'z{z}', 0) + 1
                self.total += 1

            # save reference to original geometry in tile so that we can drill
            # down later if we stop now
            tile['source'] = features

            # if it's the first-pass tiling
            if cz is None:
                # stop tiling if we reached max zoom, or if the tile is too
                # simple
                if z == options.get(
                    'indexMaxZoom') or tile.get(
                        'numPoints') <= options.get('indexMaxPoints'):
                    continue  # if a drilldown to a specific tile
            elif z == options.get('maxZoom') or z == cz:
                # stop tiling if we reached base zoom or our target tile zoom
                continue
            elif cz is not None:
                # stop tiling if it's not an ancestor of the target tile
                zoomSteps = cz - z
                if x != (cx >> zoomSteps) or y != (cy >> zoomSteps):
                    continue

            # if we slice further down, no need to keep source geometry
            tile['source'] = None

            if not features or len(features) == 0:
                continue

            logging.debug('clipping start')

            # values we'll use for clipping
            k1 = 0.5 * options.get('buffer') / options.get('extent')
            k2 = 0.5 - k1
            k3 = 0.5 + k1
            k4 = 1 + k1

            tl = None
            bl = None
            tr = None
            br = None

            left = clip(features, z2, x - k1, x + k3, 0,
                        tile['minX'], tile['maxX'], options, z+1)
            right = clip(features, z2, x + k2, x + k4, 0,
                         tile['minX'], tile['maxX'], options, z+1)
            features = None

            if left is not None:
                tl = clip(left, z2, y - k1, y + k3, 1,
                          tile['minY'], tile['maxY'], options, z+1)
                bl = clip(left, z2, y + k2, y + k4, 1,
                          tile['minY'], tile['maxY'], options, z+1)
                left = None

            if right is not None:
                tr = clip(right, z2, y - k1, y + k3, 1,
                          tile['minY'], tile['maxY'], options, z+1)
                br = clip(right, z2, y + k2, y + k4, 1,
                          tile['minY'], tile['maxY'], options, z+1)
                right = None

            logging.debug('clipping ended')

            stack.append(tl if tl is not None else [])
            stack.append(z + 1)
            stack.append(x * 2)
            stack.append(y * 2)

            stack.append(bl if bl is not None else [])
            stack.append(z + 1)
            stack.append(x * 2)
            stack.append(y * 2 + 1)

            stack.append(tr if tr is not None else [])
            stack.append(z + 1)
            stack.append(x * 2 + 1)
            stack.append(y * 2)

            stack.append(br if br is not None else [])
            stack.append(z + 1)
            stack.append(x * 2 + 1)
            stack.append(y * 2 + 1)

    def get_tile(self, z, x, y):
        z = int(z)
        x = int(x)
        y = int(y)

        options = self.options
        extent = options.get('extent')

        if z < 0 or z > 24:
            return None

        z2 = 1 << z
        x = (x + z2) & (z2 - 1)  # wrap tile x coordinate

        id_ = to_Id(z, x, y)
        current_tile = self.tiles.get(id_, None)
        if current_tile is not None:
            return transform_tile(self.tiles[id_], extent)

        logging.debug(f'drilling down to z{z}-{x}-{y}')

        z0 = z
        x0 = x
        y0 = y
        parent = None

        while parent is None and z0 > 0:
            z0 -= 1
            x0 = x0 >> 1
            y0 = y0 >> 1
            parent = self.tiles.get(to_Id(z0, x0, y0), None)

        if parent is None or parent.get('source', None) is None:
            return None

        # if we found a parent tile containing the original geometry, we can
        # drill down from it
        logging.debug(f'found parent tile z{z0}-{x0}-{y0}')
        logging.debug('drilling down start')

        self.split_tile(parent.get('source'), z0, x0, y0, z, x, y)

        logging.debug('drilling down end')

        transformed = transform_tile(
            self.tiles[id_], extent) if self.tiles.get(
                id_, None) is not None else None
        return transformed

`init(data, options, log_level=logging.INFO)`

Constructor for MicroJsonVt class

Parameters:

Name	Type	Description	Default
`data`	`dict`	The data to be converted to vector tiles	required
`options`	`dict`	The options to be used for generating vector tiles	required
`log_level`	`int`	The logging level to be used	`INFO`

Source code in src/microjson/microjson2vt/microjson2vt.py

def __init__(self, data, options, log_level=logging.INFO):
    """
    Constructor for MicroJsonVt class

    Args:
        data (dict): The data to be converted to vector tiles
        options (dict): The options to be used for generating vector tiles
        log_level (int): The logging level to be used
    """
    logging.basicConfig(
        level=log_level, format='%(asctime)s %(levelname)s %(message)s')
    options = self.options = extend(get_default_options(), options)

    # Validate and resolve tolerance_function
    tolerance_setting = options.get('tolerance_function')
    if isinstance(tolerance_setting, str):
        if tolerance_setting in AVAILABLE_TOLERANCE_FUNCTIONS:
            options['tolerance_function'] = AVAILABLE_TOLERANCE_FUNCTIONS[tolerance_setting]
        else:
            raise ValueError(
                f"Invalid tolerance function key: '{tolerance_setting}'. "
                f"Available keys: {list(AVAILABLE_TOLERANCE_FUNCTIONS.keys())}"
            )
    elif not callable(tolerance_setting):
        raise TypeError(
            "Option 'tolerance_function' must be a callable function or a valid string key."
        )
    # If it's already callable, we use it directly.

    logging.debug('preprocess data start')

    if options.get('maxZoom') < 0 or options.get('maxZoom') > 24:
        raise Exception('maxZoom should be in the 0-24 range')
    if options.get(
        'promoteId', None) is not None and options.get(
            'generateId', False):
        raise Exception(
            'promoteId and generateId cannot be used together.')

    # projects and adds simplification info
    # Create a new instance of a CartesianProjector

    features = convert(data, options)

    # Create a separate geometry for each zoom level
    for z in range(options.get('maxZoom') + 1):
        for feature in features:
            feature[f'geometry_z{z}'] = feature['geometry'].copy()

    tolerance_func = options['tolerance_function'] # Resolved above

    # Simplify features for each zoom level
    for z in range(options.get('maxZoom') + 1):
        # Calculate tolerance using the provided or default function
        tolerance = tolerance_func(z, options)
        for feature in features:
            geometry_key = f'geometry_z{z}'
            # check feature type only simplify Polygon
            if feature['type'] == 'Polygon':
                for iring in range(len(feature[geometry_key])):
                    ring = feature[geometry_key][iring]
                    # Convert geom to list of [x, y] pairs
                    coords = [[ring[i], ring[i + 1]] for i in range(
                        0, len(ring), 3)]
                    scoords = simplify(coords, tolerance)
                    # Check that it has at least 4 pairs of coordinates
                    if len(scoords) < 4:
                        # If not, use the original coordinates
                        feature[geometry_key][iring] = ring
                    else:
                        # flatten the simplified coords
                        simplified_ring = []
                        for i in range(len(scoords)):
                            simplified_ring.append(scoords[i][0])
                            simplified_ring.append(scoords[i][1])
                            simplified_ring.append(0)
                        feature[geometry_key][iring] = simplified_ring

    # tiles and tile_coords are part of the public API
    self.tiles = {}
    self.tile_coords = []

    self.stats = {}
    self.total = 0

    # wraps features (ie extreme west and extreme east)
    # features = wrap(features, options)

    # start slicing from the top tile down
    if len(features) > 0:
        self.split_tile(features, 0, 0, 0)

`split_tile(features, z, x, y, cz=None, cx=None, cy=None)`

Splits features from a parent tile to sub-tiles.

Parameters:

Name	Type	Description	Default
`features`	`list`	The features to be split	required
`z`	`int`	The zoom level of the parent tile	required
`x`	`int`	The x coordinate of the parent tile	required
`y`	`int`	The y coordinate of the parent tile	required
`cz`	`int`	The zoom level of the target tile	`None`
`cx`	`int`	The x coordinate of the target tile	`None`
`cy`	`int`	The y coordinate of the target tile	`None`

Source code in src/microjson/microjson2vt/microjson2vt.py

def split_tile(self, features, z, x, y, cz=None, cx=None, cy=None):
    """
    Splits features from a parent tile to sub-tiles.

    Args:
        features (list): The features to be split
        z (int): The zoom level of the parent tile
        x (int): The x coordinate of the parent tile
        y (int): The y coordinate of the parent tile
        cz (int): The zoom level of the target tile
        cx (int): The x coordinate of the target tile
        cy (int): The y coordinate of the target tile
    """
    stack = [features, z, x, y]
    options = self.options
    # avoid recursion by using a processing queue
    while len(stack) > 0:
        y = stack.pop()
        x = stack.pop()
        z = stack.pop()
        features = stack.pop()

        z2 = 1 << z
        id_ = to_Id(z, x, y)
        tile = self.tiles.get(id_, None)

        if tile is None:
            # Use simplified geometries for this zoom level

            simplified_features = [
                {
                    **feature,
                    "geometry": feature[f'geometry_z{z}']
                }
                for feature in features
            ]

            self.tiles[id_] = create_tile(features, z, x, y, options)
            tile = self.tiles[id_]
            self.tile_coords.append({'z': z, 'x': x, 'y': y})

            key = f'z{z}'
            self.stats[key] = self.stats.get(key, 0) + 1
            self.total += 1

            self.tiles[id_] = create_tile(
                simplified_features, z, x, y, options)
            tile = self.tiles[id_]
            self.tile_coords.append({'z': z, 'x': x, 'y': y})

            self.stats[f'z{z}'] = self.stats.get(f'z{z}', 0) + 1
            self.total += 1

        # save reference to original geometry in tile so that we can drill
        # down later if we stop now
        tile['source'] = features

        # if it's the first-pass tiling
        if cz is None:
            # stop tiling if we reached max zoom, or if the tile is too
            # simple
            if z == options.get(
                'indexMaxZoom') or tile.get(
                    'numPoints') <= options.get('indexMaxPoints'):
                continue  # if a drilldown to a specific tile
        elif z == options.get('maxZoom') or z == cz:
            # stop tiling if we reached base zoom or our target tile zoom
            continue
        elif cz is not None:
            # stop tiling if it's not an ancestor of the target tile
            zoomSteps = cz - z
            if x != (cx >> zoomSteps) or y != (cy >> zoomSteps):
                continue

        # if we slice further down, no need to keep source geometry
        tile['source'] = None

        if not features or len(features) == 0:
            continue

        logging.debug('clipping start')

        # values we'll use for clipping
        k1 = 0.5 * options.get('buffer') / options.get('extent')
        k2 = 0.5 - k1
        k3 = 0.5 + k1
        k4 = 1 + k1

        tl = None
        bl = None
        tr = None
        br = None

        left = clip(features, z2, x - k1, x + k3, 0,
                    tile['minX'], tile['maxX'], options, z+1)
        right = clip(features, z2, x + k2, x + k4, 0,
                     tile['minX'], tile['maxX'], options, z+1)
        features = None

        if left is not None:
            tl = clip(left, z2, y - k1, y + k3, 1,
                      tile['minY'], tile['maxY'], options, z+1)
            bl = clip(left, z2, y + k2, y + k4, 1,
                      tile['minY'], tile['maxY'], options, z+1)
            left = None

        if right is not None:
            tr = clip(right, z2, y - k1, y + k3, 1,
                      tile['minY'], tile['maxY'], options, z+1)
            br = clip(right, z2, y + k2, y + k4, 1,
                      tile['minY'], tile['maxY'], options, z+1)
            right = None

        logging.debug('clipping ended')

        stack.append(tl if tl is not None else [])
        stack.append(z + 1)
        stack.append(x * 2)
        stack.append(y * 2)

        stack.append(bl if bl is not None else [])
        stack.append(z + 1)
        stack.append(x * 2)
        stack.append(y * 2 + 1)

        stack.append(tr if tr is not None else [])
        stack.append(z + 1)
        stack.append(x * 2 + 1)
        stack.append(y * 2)

        stack.append(br if br is not None else [])
        stack.append(z + 1)
        stack.append(x * 2 + 1)
        stack.append(y * 2 + 1)

TileWriter module

The TileWriter module is a helper module that can be used to generate binary tiles from a large MicroJSON file, my utilizing both microjson2vt and vt2pbf.

`TileWriter`

Bases: TileHandler

Source code in src/microjson/tilewriter.py

class TileWriter (TileHandler):

    def microjson2tiles(self,
                        microjson_data_path: Union[str, Path],
                        validate: bool = False,
                        tolerance_key: str = "default"
                        ) -> List[str]:
        """
        Generate tiles in form of JSON or PBF files from MicroJSON data.

        Args:
            microjson_data_path (Union[str, Path]): Path to the
            MicroJSON data file
            validate (bool): Flag to indicate whether to validate
            the MicroJSON data

        Returns:
            List[str]: List of paths to the generated tiles
        """
        def save_tile(tile_data, z, x, y, tiles_path_template):
            """
            Save a single tile to a file based on the template path.

            Args:
                tile_data: The tile data to save
                z: The zoom level of the tile
                x: The x coordinate of the tile
                y: The y coordinate of the tile
                tiles_path_template: The template path for the tiles

            Returns:
                str: The path to the saved tile
            """
            # Format the path template with actual tile coordinates
            tile_path = str(tiles_path_template).format(z=z, x=x, y=y)
            os.makedirs(os.path.dirname(tile_path), exist_ok=True)

            # Save the tile data (this assumes tile_data is already in the
            # correct format, e.g., PBF or JSON)
            if tile_path.endswith('.parquet'):
                tile_data.to_parquet(tile_path)
            else:
                with open(
                    tile_path,
                    'wb' if tile_path.endswith('.pbf') else 'w'
                ) as f:
                    f.write(tile_data)

            # return the path to the saved tile
            return tile_path

        def convert_id_to_int(data) -> int | dict | list:
            """
            Convert all id fields in the data to integers

            Args:
                data: The data to convert

            Returns:
                dict: The data with all id fields converted to integers
            """

            # check if data is a list
            if isinstance(data, list):
                for item in data:
                    convert_id_to_int(item)
                return data
            # check if data is a dict
            elif isinstance(data, dict):
                for key, value in data.items():
                    if key == 'id':
                        if value is None:
                            data[key] = self.id_counter
                            self.id_counter += 1
                        else:
                            data[key] = int(value)
                        while data[key] in self.id_set:
                            self.id_counter += 1
                            data[key] = self.id_counter
                        self.id_set.add(data[key])
                    if isinstance(value, dict):
                        convert_id_to_int(value)
                    if isinstance(value, list):
                        for item in value:
                            convert_id_to_int(item)
                return data
            else:
                return int(data)

        # Load the MicroJSON data
        with open(microjson_data_path, 'r') as file:
            microjson_data = json.load(file)

        # Validate the MicroJSON data
        if validate:
            try:
                MicroJSON.model_validate(microjson_data)
            except ValidationError as e:
                logger.error(f"MicroJSON data validation failed: {e}")
                return []

        # TODO currently only supports one tile layer
        # calculate maxzoom and minzoom from layer and global tilejson

        maxzoom = min(self.tile_json.maxzoom,
                      self.tile_json.vector_layers[0].maxzoom)  # type: ignore
        minzoom = max(self.tile_json.minzoom,
                      self.tile_json.vector_layers[0].minzoom)  # type: ignore

        # Options for geojson2vt from TileJSON
        options = {
            'maxZoom': maxzoom,  # max zoom in the final tileset
            'indexMaxZoom': self.tile_json.maxzoom,  # tile index max zoom
            'indexMaxPoints': 0,  # max number of points per tile, 0 if none
            'bounds': self.tile_json.bounds,
            'tolerance_function': tolerance_key # Pass the string key
        }

        # Convert GeoJSON to intermediate vector tiles
        tile_index = microjson2vt(microjson_data, options)

        # Placeholder for the tile paths
        generated_tiles = []

        # get tilepath from tilejson self.tile_json.tiles
        # extract the folder from the filepath

        for tileno in tile_index.tiles:
            atile = tile_index.tiles[tileno]
            x, y, z = atile["x"], atile["y"], atile["z"]
            # if z is less than minzoom, or greater than maxzoom, skip the tile
            if z < minzoom or z > maxzoom:
                continue
            tile_data = tile_index.get_tile(z, x, y)

            for item in tile_data['features']:
                if 'id' in item:
                    item['id'] = int(item['id'])

            # add name to the tile_data
            tile_data["name"] = "tile"

            # print('tile_data before encoding')
            # print(tile_data.keys())
            # print(json.dumps(tile_data['features']))

            # convert this dictionary to a geodataframe using gpd.GeoDataFrame.from_features
            # tmp = gpd.GeoDataFrame.from_features(tile_data['features'])
            # print(tmp)

            # print('-------------------------')
            # print('-------------------------')

            # print('testing geopandas!')
            # gdf = gpd.GeoDataFrame(tile_data)
            # print(gdf)

            # print('self.pbf', self.pbf)
            # print('self.parquet', self.parquet)

            if self.pbf:
                # Using vt2pbf to encode tile data to PBF
                encoded_data = vt2pbf(tile_data)
            elif self.parquet:
                # encoded_data = pd.DataFrame(tile_data)
                encoded_data = gpd.GeoDataFrame(tile_data)

                # drop metadata columns
                encoded_data['new_geometry'] = encoded_data['features'].apply(lambda x: x['geometry'])
                # encoded_data['Label'] = encoded_data['features'].apply(lambda x: x['Label'])
                encoded_data = encoded_data[['new_geometry']]

            else:
                encoded_data = json.dumps(tile_data)

            # print('self.tile_json.tiles[0]')
            # print(self.tile_json.tiles[0])

            # print('type(encoded_data)')
            # print(type(encoded_data))

            generated_tiles.append(save_tile(
                encoded_data, z, x, y, self.tile_json.tiles[0]))

        return generated_tiles

`microjson2tiles(microjson_data_path, validate=False, tolerance_key='default')`

Generate tiles in form of JSON or PBF files from MicroJSON data.

Parameters:

Name	Type	Description	Default
`microjson_data_path`	`Union[str, Path]`	Path to the	required
`validate`	`bool`	Flag to indicate whether to validate	`False`

Returns:

Type	Description
`List[str]`	List[str]: List of paths to the generated tiles

Source code in src/microjson/tilewriter.py

def microjson2tiles(self,
                    microjson_data_path: Union[str, Path],
                    validate: bool = False,
                    tolerance_key: str = "default"
                    ) -> List[str]:
    """
    Generate tiles in form of JSON or PBF files from MicroJSON data.

    Args:
        microjson_data_path (Union[str, Path]): Path to the
        MicroJSON data file
        validate (bool): Flag to indicate whether to validate
        the MicroJSON data

    Returns:
        List[str]: List of paths to the generated tiles
    """
    def save_tile(tile_data, z, x, y, tiles_path_template):
        """
        Save a single tile to a file based on the template path.

        Args:
            tile_data: The tile data to save
            z: The zoom level of the tile
            x: The x coordinate of the tile
            y: The y coordinate of the tile
            tiles_path_template: The template path for the tiles

        Returns:
            str: The path to the saved tile
        """
        # Format the path template with actual tile coordinates
        tile_path = str(tiles_path_template).format(z=z, x=x, y=y)
        os.makedirs(os.path.dirname(tile_path), exist_ok=True)

        # Save the tile data (this assumes tile_data is already in the
        # correct format, e.g., PBF or JSON)
        if tile_path.endswith('.parquet'):
            tile_data.to_parquet(tile_path)
        else:
            with open(
                tile_path,
                'wb' if tile_path.endswith('.pbf') else 'w'
            ) as f:
                f.write(tile_data)

        # return the path to the saved tile
        return tile_path

    def convert_id_to_int(data) -> int | dict | list:
        """
        Convert all id fields in the data to integers

        Args:
            data: The data to convert

        Returns:
            dict: The data with all id fields converted to integers
        """

        # check if data is a list
        if isinstance(data, list):
            for item in data:
                convert_id_to_int(item)
            return data
        # check if data is a dict
        elif isinstance(data, dict):
            for key, value in data.items():
                if key == 'id':
                    if value is None:
                        data[key] = self.id_counter
                        self.id_counter += 1
                    else:
                        data[key] = int(value)
                    while data[key] in self.id_set:
                        self.id_counter += 1
                        data[key] = self.id_counter
                    self.id_set.add(data[key])
                if isinstance(value, dict):
                    convert_id_to_int(value)
                if isinstance(value, list):
                    for item in value:
                        convert_id_to_int(item)
            return data
        else:
            return int(data)

    # Load the MicroJSON data
    with open(microjson_data_path, 'r') as file:
        microjson_data = json.load(file)

    # Validate the MicroJSON data
    if validate:
        try:
            MicroJSON.model_validate(microjson_data)
        except ValidationError as e:
            logger.error(f"MicroJSON data validation failed: {e}")
            return []

    # TODO currently only supports one tile layer
    # calculate maxzoom and minzoom from layer and global tilejson

    maxzoom = min(self.tile_json.maxzoom,
                  self.tile_json.vector_layers[0].maxzoom)  # type: ignore
    minzoom = max(self.tile_json.minzoom,
                  self.tile_json.vector_layers[0].minzoom)  # type: ignore

    # Options for geojson2vt from TileJSON
    options = {
        'maxZoom': maxzoom,  # max zoom in the final tileset
        'indexMaxZoom': self.tile_json.maxzoom,  # tile index max zoom
        'indexMaxPoints': 0,  # max number of points per tile, 0 if none
        'bounds': self.tile_json.bounds,
        'tolerance_function': tolerance_key # Pass the string key
    }

    # Convert GeoJSON to intermediate vector tiles
    tile_index = microjson2vt(microjson_data, options)

    # Placeholder for the tile paths
    generated_tiles = []

    # get tilepath from tilejson self.tile_json.tiles
    # extract the folder from the filepath

    for tileno in tile_index.tiles:
        atile = tile_index.tiles[tileno]
        x, y, z = atile["x"], atile["y"], atile["z"]
        # if z is less than minzoom, or greater than maxzoom, skip the tile
        if z < minzoom or z > maxzoom:
            continue
        tile_data = tile_index.get_tile(z, x, y)

        for item in tile_data['features']:
            if 'id' in item:
                item['id'] = int(item['id'])

        # add name to the tile_data
        tile_data["name"] = "tile"

        # print('tile_data before encoding')
        # print(tile_data.keys())
        # print(json.dumps(tile_data['features']))

        # convert this dictionary to a geodataframe using gpd.GeoDataFrame.from_features
        # tmp = gpd.GeoDataFrame.from_features(tile_data['features'])
        # print(tmp)

        # print('-------------------------')
        # print('-------------------------')

        # print('testing geopandas!')
        # gdf = gpd.GeoDataFrame(tile_data)
        # print(gdf)

        # print('self.pbf', self.pbf)
        # print('self.parquet', self.parquet)

        if self.pbf:
            # Using vt2pbf to encode tile data to PBF
            encoded_data = vt2pbf(tile_data)
        elif self.parquet:
            # encoded_data = pd.DataFrame(tile_data)
            encoded_data = gpd.GeoDataFrame(tile_data)

            # drop metadata columns
            encoded_data['new_geometry'] = encoded_data['features'].apply(lambda x: x['geometry'])
            # encoded_data['Label'] = encoded_data['features'].apply(lambda x: x['Label'])
            encoded_data = encoded_data[['new_geometry']]

        else:
            encoded_data = json.dumps(tile_data)

        # print('self.tile_json.tiles[0]')
        # print(self.tile_json.tiles[0])

        # print('type(encoded_data)')
        # print(type(encoded_data))

        generated_tiles.append(save_tile(
            encoded_data, z, x, y, self.tile_json.tiles[0]))

    return generated_tiles

`extract_fields_ranges_enums(microjson_file)`

Extract field names, ranges, and enums from the provided MicroJSON file. Returns: tuple: (dict with field names and types, dict of field ranges, dict of field enums)

Source code in src/microjson/tilewriter.py

def extract_fields_ranges_enums(microjson_file: str):
    """
    Extract field names, ranges, and enums from the provided MicroJSON
    file.
    Returns:
        tuple: (dict with field names and types,
                dict of field ranges,
                dict of field enums)
    """
    import json

    def get_json_type(value):
        if value is None:
            # Set to String if None
            return 'String'
        if isinstance(value, bool):
            return 'Boolean'
        if isinstance(value, (int, float)):
            return 'Number'
        if isinstance(value, dict):
            return 'Object'
        if isinstance(value, list):
            return 'Array'
        return 'String'

    with open(microjson_file, 'r') as file:
        data = json.load(file)

    field_names: dict[str, str] = {}
    field_ranges = {}
    field_enums: dict[str, set[str]] = {}

    for feature in data.get('features', []):
        props = feature.get('properties', {})
        for key, val in props.items():
            if key not in field_names.keys():
                field_names[key] = get_json_type(val)
            if isinstance(val, (int, float)):
                if key not in field_ranges:
                    field_ranges[key] = [val, val]
                else:
                    field_ranges[key][0] = min(field_ranges[key][0], val)
                    field_ranges[key][1] = max(field_ranges[key][1], val)
            if isinstance(val, str):
                if key not in field_enums:
                    field_enums[key] = set()
                field_enums[key].add(val)

    return field_names, field_ranges, field_enums

`getbounds(microjson_file, square=False)`

Get the max and min bounds for coordinates of the MicroJSON file

Parameters:

Name	Type	Description	Default
`microjson_file`	`str`	Path to the MicroJSON file	required
`square`	`bool`	Flag to indicate whether to return square bounds	`False`

Returns:

Type	Description
`List[float]`	List[float]: List of the bounds [minx, miny, maxx, maxy]

Source code in src/microjson/tilewriter.py

def getbounds(microjson_file: str, square: bool = False) -> List[float]:
    """
    Get the max and min bounds for coordinates of the MicroJSON file

    Args:
        microjson_file (str): Path to the MicroJSON file
        square (bool): Flag to indicate whether to return square bounds

    Returns:
        List[float]: List of the bounds [minx, miny, maxx, maxy]
    """
    with open(microjson_file, 'r') as file:
        data = json.load(file)

    # get the bounds
    minx = miny = float('inf')
    maxx = maxy = float('-inf')
    if 'features' in data:
        for feature in data['features']:
            if 'geometry' in feature:
                if feature['geometry']['type'] == 'Polygon':
                    for ring in feature['geometry']['coordinates']:
                        for coord in ring:
                            minx = min(minx, coord[0])
                            miny = min(miny, coord[1])
                            maxx = max(maxx, coord[0])
                            maxy = max(maxy, coord[1])
                if feature['geometry']['type'] == 'MultiPolygon':
                    for polygon in feature['geometry']['coordinates']:
                        for ring in polygon:
                            for coord in ring:
                                minx = min(minx, coord[0])
                                miny = min(miny, coord[1])
                                maxx = max(maxx, coord[0])
                                maxy = max(maxy, coord[1])
    if square:
        maxx = max(maxx - minx, maxy - miny) + minx
        maxy = max(maxx - minx, maxy - miny) + miny
    return [minx, miny, maxx, maxy]

An example of how to use the TileWriter module is located in the src/microjson/examples/tiling.py file of the repository. The example demonstrates how to generate binary tiles from a large MicroJSON file.

TileReader module

Correspondingly, the TileReader module is a helper module that can be used to read binary tiles and convert them back to MicroJSON objects.

`TileReader`

Bases: TileHandler

Class to read tiles and generate MicroJSON data

Source code in src/microjson/tilereader.py

class TileReader(TileHandler):
    """
    Class to read tiles and generate MicroJSON data
    """

    def tiles2microjson(self,
                        zlvl: int = 0) -> dict[str, Any]:
        """
        Generate MicroJSON data from tiles in form of JSON or PBF files.
        Get the TileJSON configuration and the PBF flag from the class
        attributes.
        Check that zlvl is within the maxzoom and minzoom of the tilejson
        Get the bounds from the tilejson and use to generate the MicroJSON
        data, by reading the tiles at the specified zoom level and
        extracting the geometries from the tiles.

        Args:
            zlvl (int): The zoom level of the tiles to read

        Returns:
            dict: The generated MicroJSON data
        """

        # check if zlvl is within the maxzoom and minzoom of the tilejson
        if (self.tile_json.minzoom is None or
                zlvl < self.tile_json.minzoom or
                self.tile_json.maxzoom is None or
                zlvl > self.tile_json.maxzoom):
            return {}

        # get the bounds from the tilejson
        bounds = self.tile_json.bounds
        if bounds is None:
            return {}

        minx = float(bounds[0])
        miny = float(bounds[1])
        maxx = float(bounds[2])
        maxy = float(bounds[3])
        ntiles = 2 ** zlvl
        xstep = (maxx - minx) / ntiles
        ystep = (maxy - miny) / ntiles
        xstarts = [minx + x * xstep for x in range(ntiles)]
        ystarts = [miny + y * ystep for y in range(ntiles)]
        xstops = [minx + (x + 1) * xstep for x in range(ntiles)]
        ystops = [miny + (y + 1) * ystep for y in range(ntiles)]

        # reverse the ystarts and ystops
        # ystarts = ystarts[::-1]
        # ystops = ystops[::-1]

        def project(coord, xmin, ymin, xmax, ymax,
                    extent=4096):
            return [
                (coord[0] / extent * (xmax - xmin) + xmin),
                (coord[1] / extent * (ymax - ymin) + ymin)
            ]

        # get the tilepath from the tilejson
        tilepath = str(self.tile_json.tiles[0])

        # initialize the microjson data
        microjson_data = {
            "type": "FeatureCollection",
            "features": []
        }

        # read the tiles and extract the geometries
        for x in range(ntiles):
            for y in range(ntiles):
                xstart = xstarts[x]
                xstop = xstops[x]
                ystart = ystarts[y]
                ystop = ystops[y]
                # format path template with tile coordinates
                tile_file = tilepath.format(z=zlvl, x=x, y=y)

                if not os.path.exists(str(tile_file)):
                    continue

                with open(
                        str(tile_file),
                        'rb' if str(tile_file).endswith('.pbf') else 'r') as f:
                    tile_data = f.read()

                # decode the tile data
                if self.pbf:
                    tile_data = mapbox_vector_tile.decode(
                        tile_data,
                        default_options={
                            "geojson": True,
                            "y_coord_down": True})
                else:
                    tile_data = json.loads(tile_data)

                # dump to file
                # filename = f"tilevt11_{x}_{y}_{zlvl}.json"

                # with open(filename, "w") as f:
                #    json.dump(tile_data, f)

                tile_data = tile_data['geojsonLayer']

                # extract the geometries
                if 'features' in tile_data.keys():
                    for feature in tile_data['features']:
                        # Transform the coordinates to the global coordinate
                        # system please note that the coordinates may be in
                        # up to 5 nested lists transform the coordinates in
                        # place
                        if 'geometry' in feature:
                            geom = feature['geometry']
                            coord = geom['coordinates']
                            if 'type' in geom:
                                if geom['type'] == 'Point':
                                    geom['coordinates'] = project(
                                        coord, xstart, ystart, xstop, ystop)
                                elif geom['type'] == 'LineString':
                                    geom['coordinates'] = [
                                        project(coord, xstart, ystart, xstop,
                                                ystop)
                                        for coord in geom['coordinates']
                                    ]
                                elif geom['type'] == 'Polygon':
                                    geom['coordinates'] = [
                                        [
                                            project(coord, xstart, ystart,
                                                    xstop, ystop)
                                            for coord in ring
                                        ]
                                        for ring in geom['coordinates']
                                    ]
                                elif geom['type'] == 'MultiPolygon':
                                    geom['coordinates'] = [
                                        [
                                            [
                                                project(coord, xstart, ystart,
                                                        xstop, ystop)
                                                for coord in ring
                                            ]
                                            for ring in poly
                                        ]
                                        for poly in geom['coordinates']
                                    ]
                                else:
                                    continue

                            # add the feature to the microjson data
                            microjson_data['features'].append(  # type: ignore
                                feature)

        return microjson_data

`tiles2microjson(zlvl=0)`

Generate MicroJSON data from tiles in form of JSON or PBF files. Get the TileJSON configuration and the PBF flag from the class attributes. Check that zlvl is within the maxzoom and minzoom of the tilejson Get the bounds from the tilejson and use to generate the MicroJSON data, by reading the tiles at the specified zoom level and extracting the geometries from the tiles.

Parameters:

Name	Type	Description	Default
`zlvl`	`int`	The zoom level of the tiles to read	`0`

Returns:

Name	Type	Description
`dict`	`dict[str, Any]`	The generated MicroJSON data

Source code in src/microjson/tilereader.py

def tiles2microjson(self,
                    zlvl: int = 0) -> dict[str, Any]:
    """
    Generate MicroJSON data from tiles in form of JSON or PBF files.
    Get the TileJSON configuration and the PBF flag from the class
    attributes.
    Check that zlvl is within the maxzoom and minzoom of the tilejson
    Get the bounds from the tilejson and use to generate the MicroJSON
    data, by reading the tiles at the specified zoom level and
    extracting the geometries from the tiles.

    Args:
        zlvl (int): The zoom level of the tiles to read

    Returns:
        dict: The generated MicroJSON data
    """

    # check if zlvl is within the maxzoom and minzoom of the tilejson
    if (self.tile_json.minzoom is None or
            zlvl < self.tile_json.minzoom or
            self.tile_json.maxzoom is None or
            zlvl > self.tile_json.maxzoom):
        return {}

    # get the bounds from the tilejson
    bounds = self.tile_json.bounds
    if bounds is None:
        return {}

    minx = float(bounds[0])
    miny = float(bounds[1])
    maxx = float(bounds[2])
    maxy = float(bounds[3])
    ntiles = 2 ** zlvl
    xstep = (maxx - minx) / ntiles
    ystep = (maxy - miny) / ntiles
    xstarts = [minx + x * xstep for x in range(ntiles)]
    ystarts = [miny + y * ystep for y in range(ntiles)]
    xstops = [minx + (x + 1) * xstep for x in range(ntiles)]
    ystops = [miny + (y + 1) * ystep for y in range(ntiles)]

    # reverse the ystarts and ystops
    # ystarts = ystarts[::-1]
    # ystops = ystops[::-1]

    def project(coord, xmin, ymin, xmax, ymax,
                extent=4096):
        return [
            (coord[0] / extent * (xmax - xmin) + xmin),
            (coord[1] / extent * (ymax - ymin) + ymin)
        ]

    # get the tilepath from the tilejson
    tilepath = str(self.tile_json.tiles[0])

    # initialize the microjson data
    microjson_data = {
        "type": "FeatureCollection",
        "features": []
    }

    # read the tiles and extract the geometries
    for x in range(ntiles):
        for y in range(ntiles):
            xstart = xstarts[x]
            xstop = xstops[x]
            ystart = ystarts[y]
            ystop = ystops[y]
            # format path template with tile coordinates
            tile_file = tilepath.format(z=zlvl, x=x, y=y)

            if not os.path.exists(str(tile_file)):
                continue

            with open(
                    str(tile_file),
                    'rb' if str(tile_file).endswith('.pbf') else 'r') as f:
                tile_data = f.read()

            # decode the tile data
            if self.pbf:
                tile_data = mapbox_vector_tile.decode(
                    tile_data,
                    default_options={
                        "geojson": True,
                        "y_coord_down": True})
            else:
                tile_data = json.loads(tile_data)

            # dump to file
            # filename = f"tilevt11_{x}_{y}_{zlvl}.json"

            # with open(filename, "w") as f:
            #    json.dump(tile_data, f)

            tile_data = tile_data['geojsonLayer']

            # extract the geometries
            if 'features' in tile_data.keys():
                for feature in tile_data['features']:
                    # Transform the coordinates to the global coordinate
                    # system please note that the coordinates may be in
                    # up to 5 nested lists transform the coordinates in
                    # place
                    if 'geometry' in feature:
                        geom = feature['geometry']
                        coord = geom['coordinates']
                        if 'type' in geom:
                            if geom['type'] == 'Point':
                                geom['coordinates'] = project(
                                    coord, xstart, ystart, xstop, ystop)
                            elif geom['type'] == 'LineString':
                                geom['coordinates'] = [
                                    project(coord, xstart, ystart, xstop,
                                            ystop)
                                    for coord in geom['coordinates']
                                ]
                            elif geom['type'] == 'Polygon':
                                geom['coordinates'] = [
                                    [
                                        project(coord, xstart, ystart,
                                                xstop, ystop)
                                        for coord in ring
                                    ]
                                    for ring in geom['coordinates']
                                ]
                            elif geom['type'] == 'MultiPolygon':
                                geom['coordinates'] = [
                                    [
                                        [
                                            project(coord, xstart, ystart,
                                                    xstop, ystop)
                                            for coord in ring
                                        ]
                                        for ring in poly
                                    ]
                                    for poly in geom['coordinates']
                                ]
                            else:
                                continue

                        # add the feature to the microjson data
                        microjson_data['features'].append(  # type: ignore
                            feature)

    return microjson_data

An example of how to use the TileReader module is located in the src/microjson/examples/readtiles.py file of the repository. The example demonstrates how to read binary tiles and convert them back to MicroJSON objects.

TileJSON for MicroJSON example with Vector Layers

The below example illustrates a TileJSON for a MicroJSON tileset multiple layers of detail. The tileset has a single vector layer, image_layer id of vector_layers, which contains a single vector layer describing images. The fields property of the this layer specifies the attributes of the layer, including the data types of the attributes. The tiles property specifies the URL pattern for accessing the vector tiles, which in this case is a 2D data set (no channels, time or z-axis) with zoom level.

This file is located in the examples/tiles directory of the repository, and is named tiled_example.json. It has a corresponding MicroJSON file for each tile, located in the examples/tiles/tiled_example directory of the repository. The MicroJSON files are organized according to the tiling scheme, with the directory structure zlvl/x/y.json where zlvl is the zoom level, x is the x coordinate, and y is the y coordinate. The MicroJSON files contain the MicroJSON objects for the corresponding tiles, and are named according to the tiling scheme. For example, the MicroJSON object for the tile at zoom level 1, tile at (0,1) in the tiling scheme is located at examples/tiles/tiled_example/1/0/1.json. Examples for MicroJSON objects at zoom levels 0, 1, and 2 are provided below.

{
    {
    "tilejson": "3.0.0",
    "name": "2D Data Example",
    "description": "A tileset showing 2D data with multiple layers of detail.",
    "version": "1.0.0",
    "attribution": "<a href='http://example.com'>Example</a>",
    "tiles": [
        "http://example.com/tiled_example/{zlvl}/{x}/{y}.json"
    ],
    "minzoom": 0,
    "maxzoom": 10,
    "bounds": [0, 0, 24000, 24000],
    "center": [12000, 12000, 0],
    "vector_layers": [
        {
            "id": "Tile_layer",
            "description": "Tile layer",
            "minzoom": 0,
            "maxzoom": 10,
            "fields": {
              "plate": "String",
              "image": "String",
              "label": "Number",
              "channel": "Number"
            }
        }
    ],
    "fillzoom": 3,
    "multiscale": {
        "axes": [
            {
                "name": "x",
                "unit": "micrometer",
                "type": "space",
                "description": "x-axis"
            },
            {
                "name": "y",
                "unit": "micrometer",
                "type": "space",
                "description": "y-axis"
            }
        ],
        "transformationMatrix": [
            [
                1.0,
                0.0,
                0.0
            ],
            [
                0.0,
                1.0,
                0.0
            ],
            [
                0.0,
                0.0,
                0.0
            ]
        ]
    }
}

Tiled binary TileJSON

In addition to json format, tiles may be encoded in a binary protobuf format. Below follows a similar example to the one above, but with binary tiles. The tiles property specifies the URL pattern for accessing the binary tiles, which in this case is a 2D data set (no channels, time or z-axis) with zoom level. The fillzoom property specifies the zoom level from which to generate overzoomed tiles, which in this case starts at level 3, after the last specified layer.

{
    "tilejson": "3.0.0",
    "name": "2D Data Example",
    "description": "A tileset showing 2D data with multiple layers of detail.",
    "version": "1.0.0",
    "attribution": "<a href='http://example.com'>Example</a>",
    "tiles": [
        "http://example.com/tiled_example/{zlvl}/{x}/{y}.pbf"
    ],
    "minzoom": 0,
    "maxzoom": 10,
    "bounds": [0, 0, 24000, 24000],
    "center": [12000, 12000, 0],
    "vector_layers": [
        {
            "id": "tile_layer",
            "description": "Tile layer",
            "minzoom": 0,
            "maxzoom": 10,
            "fields": {
              "plate": "String",
              "image": "String",
              "label": "Number",
              "channel": "Number",
            },
            "fieldranges": {
              "label": [0,100],
              "channel": [0,10]
            },
            "fieldenums": {
              "plate": ["A1", "A2", "B1", "B2"],
              "image": ["image1.tif", "image2.tif", "image3.tif"],
            }
            "fielddescriptions": {
              "plate": "Well plate identifier",
              "image": "Image filename",
              "label": "Label identifier",
              "channel": "Channel identifier"
            }
        }
    ],
    "fillzoom": 3
}

Tiled binary example

The examples folder contains an example of how to generate binary tiles from one large MicroJSON file. It uses a helper module that generates a large random polygon grid, as could be expected in an imaging setting, using typical imaging coordinates. It is also included below for reference.

Example of creating binary tiles from a large MicroJSON file

Source code in src/microjson/examples/tiling.py

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("microjson_data_path", nargs='?',
                        default="",
                        help="Path to the MicroJSON data file")
    args = parser.parse_args()

    if args.microjson_data_path:
        microjson_data_path = args.microjson_data_path
        do_generate = False

        # Extract fields, ranges, enums from the provided MicroJSON
        field_names, field_ranges, field_enums = extract_fields_ranges_enums(
            microjson_data_path)

        # Create a TileLayer including the extracted fields
        vector_layers = [
            TileLayer(
                id="extracted-layer",
                fields=field_names,
                minzoom=0,
                maxzoom=10,
                description="Layer with extracted fields",
                fieldranges=field_ranges,
                fieldenums=field_enums,
            )
        ]
    else:
        microjson_data_path = "example.json"
        do_generate = True

    # clear the tiles directory
    os.system("rm -rf tiles")

    # create the tiles directory
    os.makedirs("tiles")

    if do_generate:
        # Create a microjson file with random polygons
        GRID_SIZE = 10000
        CELL_SIZE = 100
        MIN_VERTICES = 10
        MAX_VERTICES = 100
        meta_types = {
            "num_vertices": "int",
        }
        meta_values_options = {
            "polytype": ["Type1", "Type2", "Type3", "Type4"]
        }

        vector_layers = [
            TileLayer(
                id="polygon-layer",
                fields={"id": "String", "polytype": "String"},
                minzoom=0,
                maxzoom=10,
                description="Layer containing polygon data",
                fieldranges={
                    "num_vertices": [10, 100]
                },
                fieldenums={
                    "polytype": ["Type1", "Type2", "Type3", "Type4"]
                },
            )
        ]

        generate_polygons(
            GRID_SIZE,
            CELL_SIZE,
            MIN_VERTICES,
            MAX_VERTICES,
            meta_types,
            meta_values_options,
            microjson_data_path
        )

    # get bounds
    maxbounds = getbounds(microjson_data_path, square=True)

    center = [0,
              (maxbounds[0] + maxbounds[2]) / 2,
              (maxbounds[1] + maxbounds[3]) / 2]

    # Instantiate TileModel with your settings
    tile_model = TileModel(
        tilejson="3.0.0",
        tiles=[Path("tiles/{z}/{x}/{y}.pbf")],  # Local path or URL
        name="Example Tile Layer",
        description="A TileJSON example incorporating MicroJSON data",
        version="1.0.0",
        attribution="Polus AI",
        minzoom=0,
        maxzoom=7,
        bounds=maxbounds,
        center=center,
        vector_layers=vector_layers
    )

    # Create the root model with your TileModel instance
    tileobj = TileJSON(root=tile_model)

    # export to tilejson
    os.makedirs("tiles", exist_ok=True)
    with open("tiles/metadata.json", "w") as f:
        f.write(tileobj.model_dump_json(indent=2))

    # Initialize the TileHandler
    handler = TileWriter(tile_model, pbf=True)
    handler.microjson2tiles(microjson_data_path, validate=False)

Tiled MicroJSON Example

Level 0

The following is an example of a MicroJSON object at zoom level 0, tile at (0,0) in the tiling scheme. Example URL: http://example.com/tiles/0/0/0.json

{
    "tilejson": "3.0.0",
    "name": "2D Data Example",
    "description": "A tileset showing 2D data with multiple layers of detail.",
    "version": "1.0.0",
    "attribution": "<a href='http://example.com'>Example</a>",
    "tiles": [
        "http://example.com/tiled_example/{zlvl}/{x}/{y}.json"
    ],
    "minzoom": 0,
    "maxzoom": 10,
    "bounds": [0, 0, 24000, 24000],
    "center": [12000, 12000, 0],
    "format": "json",
    "vector_layers": [
        {
            "id": "image_layer",
            "description": "Image layer",
            "minzoom": 0,
            "maxzoom": 10,
            "fields": {
              "plate": "String",
              "image": "String",
              "label": "Number",
              "channel": "Number"
            }
        }
    ],
    "fillzoom": 3
}

Level 1

The following is an example of a MicroJSON object at zoom level 1, tile at (0,1) in the tiling scheme. Example URL: http://example.com/tiles/1/0/1.json

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [
              0,
              10000
            ],
            [
              10000,
              10000
            ],
            [
              10000,
              20000
            ],
            [
              0,
              20000
            ],
            [
              0,
              10000
            ]
          ]
        ]
      },
      "properties": {
        "label": 3
      }
    }
  ],
  "multiscale": {
    "axes": [
      {
        "name": "x",
        "type": "space",
        "unit": "micrometer",
        "description": "x-axis"
      },
      {
        "name": "y",
        "type": "space",
        "unit": "micrometer",
        "description": "y-axis"
      }
    ]
  },
  "properties": {
    "plate": "label",
    "image": "x00_y01_p01_c1.ome.tif",
    "channel": 1.0
  }
}

Level 2

The following is an example of a MicroJSON object at zoom level 2, tile at (1,1) in the tiling scheme. Example URL: http://example.com/tiles/2/1/3.json

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [
              5000,
              15000
            ],
            [
              10000,
              15000
            ],
            [
              10000,
              20000
            ],
            [
              5000,
              20000
            ],
            [
              5000,
              15000
            ]
          ]
        ]
      },
      "properties": {
        "label": 13
      }
    }
  ],
  "multiscale": {
    "axes": [
      {
        "name": "x",
        "type": "space",
        "unit": "micrometer",
        "description": "x-axis"
      },
      {
        "name": "y",
        "type": "space",
        "unit": "micrometer",
        "description": "y-axis"
      }
    ]
  },
  "properties": {
    "plate": "label",
    "image": "x00_y01_p01_c1.ome.tif",
    "channel": 1.0
  }
}

Integration of TileJSON with MicroJSON

Purpose

Background of TileJSON

TileJSON for MicroJSON Object Structure

Multiscale Object

Axis Object

Pydantic Model for TileJSON for MicroJSON

TileJSON

TileModel

TileLayer

General tiling requirements

MicroJSON2vt

__init__(data, options, log_level=logging.INFO)

split_tile(features, z, x, y, cz=None, cx=None, cy=None)

TileWriter module

TileWriter

microjson2tiles(microjson_data_path, validate=False, tolerance_key='default')

extract_fields_ranges_enums(microjson_file)

getbounds(microjson_file, square=False)

TileReader module

TileReader

tiles2microjson(zlvl=0)

TileJSON for MicroJSON example with Vector Layers

Tiled binary TileJSON

Tiled binary example

Example of creating binary tiles from a large MicroJSON file

Tiled MicroJSON Example

Level 0

Level 1

Level 2

`init(data, options, log_level=logging.INFO)`

`split_tile(features, z, x, y, cz=None, cx=None, cy=None)`

`TileWriter`

`microjson2tiles(microjson_data_path, validate=False, tolerance_key='default')`

`extract_fields_ranges_enums(microjson_file)`

`getbounds(microjson_file, square=False)`

`TileReader`

`tiles2microjson(zlvl=0)`