There is a long list of features currently under development for TileDB, which will appear in the
next releases. Below we list the most important ones.
Locality-sensitive Distributed Platform
We are currently developing a distributed platform for TileDB, where users can transparently place
their array data on multiple machines, while being able to easily write scalable applications
that take advantage of data locality and fault-tolerance.
High-Performance Linear Algebra
We are planning to integrate popular math libraries into our new locality-sensitive distributed
platform, in order to support Linear Algebra operations on massive datasets with unprecedented performance.
In the future, the users will be able to transfer TileDB data from one machine architecture to another.
In the next release, the iterators will ignore the deleted entries, making it easier for the user
to handle deletions. Moreover, the consolidation will completely discard the deleted entries.
Selective Fragment Consolidation
The user will soon be able to select a subset of fragments to consolidate. This will make it easy
to construct mechanisms that balance and amortize the update cost over multiple batch-writes.
The batch-write update technique makes it easy for TileDB to support versioning, i.e., a no-overwrite
principle for handling updates, which enables the user to time-travel over the various updates
taken place throughout the lifetime of an array. We will enable this feature soon.
Due to the batch-write update technique, it is very easy to undo recent writes (e.g., by deleting the
respective fragments prior to consolidation). We will enable this feature in the next version.
The user will soon be able to retile an array, e.g., by changing the tile extents or the cell/tile order.
This is useful when fine-tuning the performance of TileDB for a given application.
We are interested in providing interfaces to R, Java, Julia, Python, and Scala, as well as
integrate with distributed platforms like Spark.
Efficient In-Memory Indexing
We are planning to build efficient index structures (e.g., R*-Tree), which will be useful mainly for
the case of sparse arrays and specifically the ones sorted on the Hilbert order.
We will soon post our benchmark results for numerous settings versus several competing solutions.