Open source and free.
Zed is a system that makes data easier by utilizing our new super-structured data model.
Zui ("zoo-ee") is a desktop app to explore, query, and shape the data in your super-structured data lake.
Inspired by the Unix-tools design pattern, everything Zui does can be run from the command line. Think of Zui's components like Lego blocks that you can easily interconnect and assemble. It’s all open source, so have a look.
Zui is packaged as a desktop app, built with Electron just like Slack. Once installed, you can drag your data in or use the CLI tools to load data into a Zed lake. The Zed language, embedded in Zui, has operators to explore the shape of your data and develop low-code "shapers" to transform messy data into clean models.
Zui provides a rich query language that allows you to easily perform simple, intuitive keyword searches while running rich analytics with sophisticated filters and pipelines. Zed is available both in the Zui editor as well as in the Zed CLI tools.
Does the world really need another data format? Yes indeed! Zui stores all its data in the new data format ZNG. ZNG is richly typed and embeds its type schemas in the data stream. It's inspired by Avro, but without the hassle of out-of-band schema definitions and schema registries.
The Zui backend is built from the ground up using Go, the system language that powers massive infrastructure at the world's largest Internet sites. In fact, when Zui runs on your desktop, there's a miniature Zed server running in the background.
Your data is everything. If you're running machine-learning experiments or just exploring the schemas in your pipeline, Python is your go-to environment. But when you spend all your time futzing with JSON, waiting for it to load, or cramming messy data sets into a relational database, you realize that you're not getting stuff done. Zed turns this all around and makes managing data easy. With Zed, you can spend your time doing real work instead of futzing with your eclectic data.
Zui was originally built as a security platform but we hit a wall trying to proficiently support both search and analytics equally well for eclectic security data. After poring over all the existing tech stacks and data formats, we realized there must be a better way and stumbled upon the concept of super-structured data. Zui is still a great solution for security use cases, but we've been busy generalizing the system to handle a broad range of thorny data engineering use cases.
The Zed Lake is hot off the presses. While very much a work in progress, it's ready for you to kick the tires.
With a Zed Lake, your data can enjoy the structured-ness of relational tables while simultaneously embracing the adaptability and schemaless-ness of JSON. No need to define tables or schemas. Just load data into your Zed Lake using Git-like commits.
We're not trying to make data lakes run 10X faster. We're trying to make them 10X easier!
Whether you explore a Zed Lake with the Zui app or with the zed command-line client, we are maniacally focused on ease of use. If you find anything confusing or difficult, let us know and we'll work with you to fix it!
Please check out our Zed Lake tooling and give us feedback.
Whether you love our ideas or are skeptical, Zui and Zed are easy to try. One of our users told us: "Once I get my data into Zed, everything is easy." Seeing is believing, so give it a try.
And, please reach out to us. We'd love your feedback and ideas.