March 29, 2024

Embedded Database Systems

Curated News and Information About Commercial and Open Source Embedded Database Systems

Open Source Embedded Database Systems

Apache Derby

Derby is an embeddable SQL engine written entirely in Java. Fully transactional, multi-user with a decent SQL subset, Derby is a mature engine and freely available under the Apache license and is actively maintained. It is also distributed as part of Oracle’s Java SE Development Kit (JDK) under the name of Java DB.

Author: The Apache Software Foundation
License: APL
Features: SQL; File- or in-memory-based; Transactions; Cursors;
Pluggable types, routines, and aggregates; Extensive security; JPMS modules
Platforms: All JVM implementations
Languages: Java

BerkeleyDB

Berkeley DB is a fast, open-source embedded database and is used in several well-known open-source products, including the Linux and BSD Unix operating systems, Apache Web server, OpenOffice productivity suite. Implements B+tree and hash table indices. Implemented in C, open source.

Author: Initially SleepyCat, now Oracle
License: Affero GPL v3; Commercial licenses available
Features: File- or in-memory-based; Transactions; Cursors; Concurrency; Compression; Encryption; High Availability
Platforms: Unix/Linux/MacOSX, Windows, Android, iOS and many others
Languages: C/C++, Java, .NET, Erlang, Ruby, PHP, Perl and many others

DBreeze

DBreeze Database is a professional, open-source, embedded .NET family multi-paradigm (embedded Key-Value store, objects, NoSql, text search, multi-parameter search etc.), multi-threaded, transactional and ACID-compliant data management system.

Author: [dbreeze.tiesky.com] / Alexey Solovyov, Ivars Sudmalis
License: BSD 2-Clause
Features: ACID, multi-paradigm, No-SQL, Transactional, text search, multi-parameter search
Platforms: .NET 3.5> / Xamarin MONO Android iOS / .NET Core 1.0> / .NET Standard 1.6> / Universal Windows Platform / .NET Portable / CoreRT (Windows, Linux)
Languages: C# .NET

DuckDB

DuckDB is an embedded database designed to execute analytical SQL queries fast while embedded in another process. DuckDB’s vectorized execution engine allows for efficient large-scale data processing under memory constraints. DuckDB has bindings for R, Python, Java and others. DuckDB is written entirely in C++11 and has no external dependencies for maximal portability.

Author: Maintained by Mark Raasveldt & Hannes Mühleisen along with various other contributors
License: MIT
Features: OLAP SQL, Vectorized Execution Engine, Single file data storage, ACID MVCC transaction support, Concurrent query processing, Built-in analytical functions, Fast data transfer to host environment
Platforms: Windows, Linux, OSX, Solaris, BSD and more, only requires C++11 compiler
Languages: Written in C++ with bindings for Python, R, Java, C and others

Firebird

Firebird is derived from Borland InterBase 6.0 source code. It is open source and free even for commercial use. Its technology has been in use for more than 30 years, which makes it a very mature and stable product. Firebird is written in C++ and has been actively developed since its creation. It offers an embedded version with the same capabilities of the full server version.

Author: Initially Jim Starkey (original author of InterBase) and now the Firebird Project
License: IPL (InterBase Public License) and the IDPL (Initial Developer’s Public License). The first one covers the parts of the source code that were inherited from InterBase; the second applies to the additions and improvements made by the Firebird Project.
Features: ACID, full support of stored procedures and triggers, referential integrity, multi generational architecture, very small footprint, little or no need for specialized DBAs, almost no configuration needed, native support for all major operating systems, monitoring tables, TraceApi, cursors.
Platforms: Windows, Unix/Linux, MacOS and Android
Languages: C/C++, Delphi/FreePascal, .NET, Java, Python, PHP, Perl, Ruby, NodeJS, ODBC and others.

HyperGraphDB

HyperGraphDB is a general purpose, extensible, portable, distributed, embeddable, open-source data storage mechanism. It is a graph database designed specifically for artificial intelligence and semantic web projects, it can also be used as an embedded object-oriented database for projects of all sizes.

Author: Kobrix Software, Inc.
License: Apache 2.0
Features: Powerful data modeling and knowledge representation. Graph-oriented storage. N-ary, higher order relationships (edges) between graph nodes. Graph traversals and relational-style queries. Customizable indexing. Customizable storage management. Extensible, dynamic DB schema through custom typing. Out of the box Java OO database. Fully transactional and multi-threaded, MVCC/STM. Non-blocking concurrent writes and reads! P2P framework for data distribution.
Platforms: Java Platform, works on Linux/Windows/Mac
Languages: Any JVM based language, e.g. Java, Scala, Groovy etc.

KeyDB

KeyDB is an open source, high performance database compatible with the Redis™ API and focused on multithreading, memory efficiency and high throughput. KeyDB is able to scale vertically as well as horizontally with several built in persistence options. Professional options are available that extend the database from memory into FLASH storage to support large datasets that exceed memory limits.

Author: EQ Alpha Technology Ltd.
License: BSD, Commercial licenses available
Features: Multithreading, Active Replication, Multi Master, Subkey Expires, CRON, FLASH, MVCC
Platforms: Linux
Languages: C, C#, C++, Clojure, Crystal, D, Dart, Elixir, Erlang, Fancy, Go, Haskell, Haxe, Java, JavaScript (Node.js), Lisp, Lua, MatLab, Objective-C, OCaml, Pascal, Perl, PHP, Prolog, Pure Data, Python, R, Rebol, Ruby, Rust, Scala, Scheme, Smalltalk, Swift, Tcl, Visual Basic

Kyoto Cabinet

Fast and mature key/value store with sorted (Btree) and unsorted (Hash table) databases. Implemented in C, open source.

Author: Mikio Hirabayashi, FAL labs
License: GPL, LGPL; Commercial licenses available
Features: Transactions; Cursors
Platforms: Unix/Linux/MacOSX, Windows
Languages: C/C++, Java, .NET, Ruby, PHP, Perl and many others

LevelDB

An ordered key/value store created by Google as a lightweight implementation of the BigTable storage design. Features a write-optimised LSM tree. Google distributes LevelDB under the New BSD License. Implemented in C++.

Author: Initially Google, now the LevelDB community
License: BSD
Features: Compression; Cursors
Platforms: Unix/Linux/MacOSX, Windows
Languages: C/C++, Java, .NET, Erlang, Ruby, PHP, Perl and many others

MonetDB/e

MonetDB/e is the embeddable version of the pioneering columnar database system MonetDB. It is a fully transactional multi-user, parallel SQL engine designed for analytical workloads.

Author: Initially CWI (Netherlands, www.cwi.nl), since 2008 MonetDB B.V.  (Netherlands, www.monetdb.org)
License: MPL-2.0
Features: Optimized for in-memory processing using all available cores; memory-mapped database storage; dictionary encoded string columns; knobless, auto-indexing; hot-snapshots; JSON; UUID; Windowing functions,
analytical functions (GROUPING SETS, CUBE, ROLLUP)
Platforms: Unix/Linux/MacOSX, Windows
Languages: C, C++, Java, JavaScript (Node.js), Perl, PHP, Python, Ruby, JDBC, ODBC

Perst

An object-oriented database system product in pure Java and pure C# versions. Perst gives developers the ability to sort, store and retrieve objects (e.g. Plain Old Java Objects, POJOs) in their applications with maximum speed and with low memory and storage overhead, while leveraging the object-oriented paradigm of Java and C#.

Author: McObject
License: GPL; Commercial licenses available
Features: In-memory & persistent; Replication; ACID
Platforms: JVM, .NET, J2EE, J2M,
Languages: Java, C#

RocksDB

Built on LevelDB, with improved scalability for servers with many CPU cores. API and functionality is similar to LevelDB. Written in C++.

Author: Facebook
License: BSD
Features: Compression; Cursors
Platforms: Unix/Linux/MacOSX, Windows
Languages: C/C++, Java, .NET, Erlang, Ruby, PHP, Perl and many others

Sophia

Very fast hybrid (Ram and Disk) key/value store. Automatically moves read-intensive data to RAM or flash storage; uses disk for cold data. Implemented in C.

Author: Dmitry Simonenko (@pmwkaa)
License: BSD
Features: Transactions; Cursors
Platforms: Unix/Linux/MacOSX
Languages: C/C++, Java, .NET, Erlang, Ruby, PHP, Perl and many others

SQLite

A software library that implements a self-contained, server-less, zero-configuration, transactional SQL database engine. SQLite is the most widely deployed SQL database engine in the world. The source code is in the public domain.

Author: The SQLite developer community
License: Public Domain
Features: SQL; File- or in-memory-based; Transactions; Cursors
Platforms: Unix/Linux/MacOSX, Windows, Android, iOS and many others
Languages: C/C++, Java, .NET, Erlang, Ruby, PHP, Perl and many others

Symas Lightning MDB (LMDB)

Compact memory-mapped database, originally written for the OpenLDAP project. API is modeled after BerkeleyDB. Implements B+trees with multiversion concurrency control (MVCC), Single level store, Copy on write and provides full ACID transactions with no deadlocks.

Author: Symas Corp.
License: OpenLDAP Public License
Features: Transactions; Cursors; Concurrency; Hot Backups
Platforms: Unix/Linux/MacOSX, Windows, Android, iOS
Languages: C/C++, Java, .NET, Erlang, Ruby, PHP, Perl and many others

TileDB

TileDB is an embeddable library for storing and accessing dense and sparse multi-dimensional arrays. Arrays are ideal for efficiently modeling data that arise in advanced analytics and data science applications, from dataframes, to genomic variants, to LiDAR, to imaging and more. TileDB is cloud-optimized with data versioning and time-traveling built-into its novel format. It comes with numerous APIs and integrations with Spark, Dask, MariaDB, PrestoDB, PDAL and GDAL.

Author: TileDB, Inc.
License: MIT
Features: Support for both dense and sparse arrays; support for dataframes and key-values (via sparse arrays); Cloud storage (AWS S3, Google Cloud Storage, Azure Blob Storage); Multiple compression, encryption and checksum filters; Fully multi-threaded with parallel IO; Rapid updates, data versioning, time traveling; Array metadata and groups; Numerous APIs and integrations.
Platforms: Linux, OSX, Windows
Languages: Written in C++ with bindings for C, C++, Python, R, Java and Go

UnQLite

A fast transactional key/value database with a builtin JSON document store. Includes a scripting language to access and modify the documents.

Author: Symic Systems
License: BSD; Compression and encryption require a commercial license
Features: File- or in-memory-based; Transactions; Cursors; JSON document store
Platforms: Unix/Linux/MacOSX, Windows
Languages: C/C++, Java, .NET, Ruby, PHP, Perl and many others

upscaledb

High performance key/value store with a parameterized B+-tree. Keys and records are “typed” (i.e. 32bit integers, floats, variable length or fixed length binary data). Has built-in query language to aggregate and filter data.

Author: Christoph Rupp (@cruppstahl)
License: GPLv2 or later, with many exceptions; Commercial licenses available
Features: File- or in-memory-based; Transactions; Cursors; Compression; Encryption; Remote access; Built-in analytical functions
Platforms: Unix/Linux/MacOSX, Windows, Android, iOS
Languages: C/C++, Java, .NET, Erlang, PHP, Python

WiredTiger

WiredTiger is a production quality, high performance, scalable, NoSQL, Open Source extensible platform for data management. WiredTiger is developed and maintained by MongoDB, Inc., where it is the principal database storage engine.

WiredTiger supports row-oriented storage (where all columns of a row are stored together), column-oriented storage (where columns are stored in groups, allowing for more efficient access and storage of column subsets) and log-structured merge trees (LSM), for sustained throughput under random insert workloads.

WiredTiger includes ACID transactions with standard isolation levels and durability at both checkpoint and commit-level granularity.

WiredTiger can be used as a simple key/value store, but also has a complete schema layer, including indices and projections.

WiredTiger should be generally portable to any 64-bit system supporting the ANSI C11 and POSIX 1003.1 standards.

Author: MongoDB, Inc.
License: GPL v2 or later; Commercial licenses available
Features: Transactions and durability; Row- and Column-store; Schema including indices and projections
Platforms: Unix/Linux/MacOSX, Windows
Languages: C/C++, Python