Wednesday, 14 April 2021

Observations on Dgraph

Recently I created a POC web app based on Dgraph database. I used Svelte for front-end and Svelte-Apollo for GraphQL communication. You can find more information about the POC here.

In this post I am documenting my observations and challenges I faced while building the POC.

Development Experience 

Setting up the development environment for Dgraph, Svelte, and Svelte-Apollo is comparatively easy. I was able to resolve any hiccups with the provided documentation.

Dgraph

I used Dgraph's Slash cloud service instead of setting it up locally. The cloud service has very nice user interface and handy tools. These were missing in the local Dgraph setup. The cloud service provides query generator which is very convenient. It makes writing GraphQL statements a breeze.

Svelte

Setting up Svelte is very straight forward. I just followed the instructions provided in the website and it worked without any hiccups.

Svelte-Apollo

The initial library I used for GraphQL communication is urql. But urql has a major bug. It is not possible to implement refresh feature with the default caching mechanism. The fix for this was to change the caching mechanism. But then the development with urql will not be straightforward anymore. So I switched to Svelte-Apollo.

Setting up Svelte-Apollo was easy but development was a bit challenging. Svelte-Apollo is built on top of Apollo GraphQL library. Even though Apollo GraphQL library has very good documentation; Svelte-apollo has only one page documentation which is it's Github readme file. There is not much of choice when choosing a GraphQL library when working with Svelte. So I persisted and eventually got it working. 

Challenges

I feel most of the challenges I faced are because of the features of the architecture elements. In this POC the main architecture element is Dgraph. Its features are that it is a graph database. And it uses GraphQL for external communication.

Query language

Traditional SQL cannot be used with Dgraph. We have to use a modified GraphQL called DQL. Another query system called RDF can be used. From the comments on Dgraph forum; it says it doesn't implement the complete RDF Spec. If I wanted to use it it is another thing I will have to learn to use Dgraph.

Dgraph cloud service provides a GraphQL generator. I found myself using it often and it makes writing GraphQL queries a breeze. You can also instantly test it out also.

Foreign Keys

Dgraph foreign keys are not like traditional foreign keys. It doesn't support cascade on delete feature.

In the POC the feature that I wanted to implement was when you delete a task then all it's subtasks will be also be deleted. And when you delete a project all it's tasks and subtasks will also be deleted.

This is very easy implement in traditional database with the cascade on delete feature. To implement this feature in Dgraph first you have identify all the items that needs to deleted. You need all the id's of these items. The use the delete task/subtask/project query to delete all the items. GraphQL allows us to pass multiple delete statements in one query. So one request is sufficient to delete all the nodes.

Stored Procedures

Dgraph doesn't support stored procedures. So Business logic cannot be written in database. The business logic then should be written in the front-end or middleware. 

One plus point when using GraphQL is that you can combine multiple update/delete/insert statements.

Aggregation queries

Aggregation queries are supported in Dgraph. But it is very limited.

In the POC app I wanted to show task count for each project and subtask count for each task. In the traditional system I would have used an inner query to get the children count. I couldn't find a way to do this. Not sure if this is possible in GraphQL.

We will need to implement it as a separate service. And for every change we will have to send two requests for page refresh. First to get the updated tasks and then to get the count data.

I tried to overcome this by adding a count column for each project and task. But I found this to be very buggy. Instantly adding multiple tasks can mess up the count very easily.

Constraint Checks

Constraints checks cannot be set in Dgraph. For example unique constraints. Unique constraint is a very common constraint used in traditional database. If you want to ensure uniqueness in your data in Dgraph then you have to set put that logic wherever you put the business logic. But this can get very buggy if data is inserted from multiple sources. 

Advantages of Dgraph 

  • Your frontend can communicate with Dgraph directly using GraphQL. 
  • Dgraph is also designed to be highly scalable. 
  • Dgraph can be easily integrated with an auth system. 

Conclusion

I feel Dgraph can be successfully used for applications which just need CRUD operations. If your application has lot of business logic then it is not recommended. Although you can use Dgraph as a support database. Dgraph is a highly performant graph database. If you have a highly interconnected dataset then Dgraph is can be used to parse that data.

Edit Log:

  • Added links.

Friday, 2 April 2021

My Dgraph based POC Web App


POC Snapshot. Left "Basic App", Right "Subscribe App"

I built a POC Web App based on Dgraph. I Used Svelte for front-end. And Used Svelte-Apollo for Graphql communication with Dgraph. The middleware is missing here because Dgraph allows direct access to it from font-end using Graphql. 

You can find the source at this github repo.

Why POC?

The aim of this POC is to evaluate the challenges that a web developer will face when developing a Web App with Dgraph as a database.

About the POC App

The app is a task tracker. You can add tasks under projects. You can also add subtasks under tasks. The whole thing is shown in the hierarchical structure.

Some features of the App

  • It shows count of number of tasks for each project.
  • It shows count of number of subtasks for each task.
  • Deleting a project will also delete all its tasks and subtasks.
  • Deleting a task will also delete all its subtasks.
  • Options that can be performed on each element showup when you hover over than element.

App UI

There are two pages called Basic App and Subscribe App. Both have the same user interface but implemented differently.

Basic App

All the requests are HTTP requests. All the page requests and data manipulation requests are HTTP request.

Subscribe App

At the start the app subscribes with Dgraph using WebSockets. When the Dgraph data changes the app will receive updates. The data manipulation requests are done using HTTP requests. 

The other difference between Basic App and Subscribe App is that in Basic App after each data manipulation request we need to send HTTP request to get updated data.

Note

Currently the app doesn't allow user to enter data. Whenever user clicks on Add Project/Add Task/Add SubTask; data is inserted with mock data. The image at the beginning of the article shows the mock data. The mock data structure is <Project/Task/SubTask> <count>. The count is unique only for the browser. Everytime you open a new session the count becomes 1. After each addition the count is incremented.

Wednesday, 17 March 2021

Golang. My next choice.

For sometime I wanted to learn a programming language other than Java and SQL. These are the programming languages that I have extensively worked with. 

Java is one of the language you first learn in you academic courses. It is easy to teach and learn. And It's tools are available for free. It follows a very popular paradigm which the academics love. 

But I felt Java very verbose. It takes a lot code to implement the same functionality in Java compared to other programming languages. It is also slow to execute and consumes a lot of system resource. 

And SQL is a programming language that can be used to write only database queries.

Among the thousands of programming language choices the Rust programming language was the hot topic. From the praises I read; this was an amazing programming language. It was my default pick.

So I tried to learn Rust. But I found the syntax weird. The macro system was powerful but you have to explicitly indicate that you are calling a macro with ! symbol. As a programmer you should not only know which function to call but also be aware if it is macro or not. 

When I was looking into Haskell I loved functional programming style of writing code. But the Rust adaption of this is very inconvenient to use. Somehow the simplicity of functional programming style in Haskell is lost in Rust.

Then I tried golang. 

I was avoiding this programming language because of its garbage collection feature. But when I started looking into it I found it very easy to learn and simple. The simplicity allows you to write very terse code. 

One killer feature is the multi-threading support. The go threads are very light weight and you can spawn thousands of them very easily. Various inter process communication features are also very helpful. The multi-threading syntax is baked into the golang compiler. It is very easy to write multi-threading applications in golang.

The ability of golang to generate standalone executable is the most underrated feature in my opinion.

And as for the garbage collection its very fast. From the history various releases we can see that the garbage collection is continuously worked on and had been made fast.

Friday, 12 March 2021

DGraph. Graph database for win.

Somebody asked me look into this one. Although I am now a database developer I never showed any interest in Graph databases. I thought Graph databases are very different from normal databases. The can only be used for things like maps or social network.

When I looked into DGraph I found that we can use it for general purpose database programming too. The schema declaration is nice. Its declaration format is better than relational databases. The GraphQL integration is very convenient. You frontend can directly talk with database with GraphQL. DGraph is also very performant compared to other Graph databases.

The horizontal scalability feature of DGraph is what sets it apart from other databases. Most databases are vertically scale-able. DGraph promises it to be horizontally scale-able. So that means DGraph as a backend makes sense cost wise and also performance wise.

One con is that we cannot write stored procedures with dgraph. So for companies that write business logic in stored procedures DGraph will not be useful. Although I found one DGraph demo showcasing a different approach for handling business logic.

Svelte. WebApp development made easy.

The thing with the most popular frameworks like React, Vue, or Angular is that they are too big. Learning curve is too big learn. Cumbersome to develop apps. So I was looking for a simple framework for building webapps. No seriously. Even though there are a ton of frameworks out there I was searching out there; expecting to find nothing.

Luckily I found one. It's called Svelte. I found blown away by the tech talk by Rich Harris. You should see it too.


Svelte is not a framework but a compiler. The advantage is that the output Javascript is optimized for performance and size. 

I found a lot of other features like components, reactivity features, and small code size very enticing. This is one framework for lookout for. 

Saturday, 2 January 2021

How to install Elixir in Ubuntu or PopOS 20.10 so it works with ElixirLS

Elixir can be installed from default aptitude (apt) resources. But this install doesn't work with latest Visual Studio Code's ElixirLS plugin. The ElixirLS plugin keeps crashing.

For the ElixirLS plugin to work correctly install Elixir using the asdf tool. 

Following below link to install asdf, Erlang, and Elixir.

https://thinkingelixir.com/install-elixir-using-asdf/#What_is_asdf

asdf will build Erlang from source. So install all the software required for building it. See link below for more info.

https://github.com/asdf-vm/asdf-erlang