Persisting Data in Service Weaver Applications

In my previous post, Monolith or Microservices, or Both: Building Modern Distributed Applications in Go with Service Weaver, I have given a brief introduction about Service Weaver, a programming framework that makes it easy to write, test, deploy, and manage distributed applications in Go. Service Weaver lets you write Go application as a modular monolith, which can be deployed it as either monolith or microservices by using the Service Weaver runtime. In this post, I will take a look at how to persist data in Service Weaver applications. The source code of the example demo is available on github at: https://github.com/shijuvar/service-weaver/tree/main/orderapp.

Storage in Service Weaver Applications

Many modern distributed application frameworks and runtimes provide native support for storage and persistence. For example, Encore, a distributed application framework in Go, natively supports SQL databases for storage by using PostgreSQL database, and thus it provides native support for persistence and migrations. This provides better portability when you move your applications from one environment to another one. But at the same time, this has a lot of limitations. First and foremost, you stuck with a particular opinionated storage. When you build modern applications, you may use variety of storage options from NoSQL Databases, SQL Databases, Distributed SQL Databases like CockroachDB, etc.

Service Weaver doesn’t provide native support for storage

Service Weaver doesn’t provide native support for storage so that you can choose your choice of databases based on the context of your application. In my opinion, this is better choice over the native support for storage, although it makes some portability concerns. I don’t prefer to use a storage option for all kind of applications recommended by a framework although it may provide better portability and developer productivity. And thus, I can implement persistence and migration in my own way. There is no perfect solution in software architetcure.

Using Service Weaver config files for specifying storage systems for different environments

Although Service Weaver doesn’t provide native support for storage, one option we can do is leverage the Service Weaver config files to specify the storage options for different environments by integrating with WithConfig[T]. Service Weaver type WithConfig[T] can be embedded inside a component implementation that lets the components automatically take configuration information found in the application config file and use it to initialize the contents of T.

Using Service Weaver Components for Persistence

Let’s write a simple example demo for persisting data in Service Weaver applications. Here we use persistence component as Service Weaver components to leverage the capabilities of Service Weaver runtime. Let’s start by writing an interface for the persistence component:

Listing 1. Interface for the Service Weaver component for persistence and data retrieval, in the package cockroachdb

// Repository interface for command and query operations
type Repository interface {
 CreateOrder(context.Context, model.Order) error
 GetOrderByID(context.Context, string) (model.Order, error)
 GetOrderItems(context.Context,string) ([]model.OrderItem, error)
}

In our concrete implementation for the Repository interface, we access the storage information from the Service Weaver config file by embedding with weaver.WithConfig[T] struct.

Here’s the Service Weaver config file:

Listing 2. Service Weaver config file, weaver.toml

[serviceweaver]
binary = "./orderapp"

[single]
listeners.orderapi = {address = "localhost:3000"}

[multi]
listeners.orderapi = {address = "localhost:3000"}

[gke]
regions = ["us-west1"]
listeners.orderapi = {public_hostname = "orderapp.example.com"}

["github.com/shijuvar/service-weaver/orderapp/cockroachdb/Repository"]
Driver = "postgres"
Source = "postgresql://shijuvar@localhost:26257/ordersdb?sslmode=disable"

In the preceding configuration file, we have provided configuration options for Driver for specifying the database driver and Source for specifying the databases’ source URI. Because we are implementing the component for the Repository interface of github.com/shijuvar/service-weaver/orderapp/cockroachdb package, these storage options are provided in the section [“github.com/shijuvar/service-weaver/orderapp/cockroachdb/Repository”].

Let’s provide a concrete implementation for the Repository interface, by embedding weaver.Implements[Repository], and also embedding weaver.WithConfig[T] to be integrated with Service Weaver config files.

Listing 3. Implementation for the Repository interface working with a CockroachDB database

package cockroachdb

import (
 "context"
 "database/sql"
 "errors"

 "github.com/ServiceWeaver/weaver"
 "github.com/cockroachdb/cockroach-go/crdb"
 _ "github.com/lib/pq"

 "github.com/shijuvar/service-weaver/orderapp/model"
)

// Implementation for Repository interface
type repository struct {
 weaver.Implements[Repository]
 weaver.WithConfig[config]
 db *sql.DB
}

// Init access storage information from configuration and connect to SQL DB
func (repo *repository) Init(context.Context) error {
 cfg := repo.Config()
 if err := cfg.Validate(); err != nil {
  repo.Logger().Error("error:", err)
 }
 db, err := sql.Open(cfg.Driver, cfg.Source)
 repo.Logger().Info("connected to DB")

 if err != nil {
  return err
 }
 repo.db = db
 return nil
}

// CreateOrder persist Order data 
func (repo *repository) CreateOrder(ctx context.Context, order model.Order) error {

 // Run a transaction to sync the query model.
 err := crdb.ExecuteTx(ctx, repo.db, nil, func(tx *sql.Tx) error {
  return createOrder(tx, order)
 })
 if err != nil {
  return err
 }
 return nil
}

func createOrder(tx *sql.Tx, order model.Order) error {

 // Insert into the "orders" table.
 sql := `
   INSERT INTO orders (id, customerid, status, createdon, restaurantid, amount)
   VALUES ($1,$2,$3,$4,$5,$6)`
 _, err := tx.Exec(sql, order.ID, order.CustomerID, order.Status, order.CreatedOn, order.RestaurantId, order.Amount)
 if err != nil {
  return err
 }
 // Insert items into the "orderitems" table.
 // Because it's store for read model, we can insert denormalized data
 for _, v := range order.OrderItems {
  sql = `
   INSERT INTO orderitems (orderid, code, name, unitprice, quantity)
   VALUES ($1,$2,$3,$4,$5)`

  _, err := tx.Exec(sql, order.ID, v.ProductCode, v.Name, v.UnitPrice, v.Quantity)
  if err != nil {
   return err
  }
 }
 return nil
}

// GetOrderByID query the Orders by given id
func (repo *repository) GetOrderByID(ctx context.Context, id string) (model.Order, error) {
 var orderRow = model.Order{}
 if err := repo.db.QueryRowContext(ctx,
  "SELECT id, customerid, status, createdon, restaurantid FROM orders WHERE id = $1",
  id).
  Scan(
   &orderRow.ID, &orderRow.CustomerID, &orderRow.Status, &orderRow.CreatedOn, &orderRow.RestaurantId,
  ); err != nil {
  return orderRow, err
 }
 return orderRow, nil
}

// GetOrderItems query the order items by given order id
func (repo *repository) GetOrderItems(ctx context.Context, id string) ([]model.OrderItem, error) {
 rows, err := repo.db.QueryContext(ctx,
  "SELECT code, name, unitprice, quantity FROM orderitems WHERE orderid = $1", id)
 if err != nil {
  return nil, err
 }
 defer rows.Close()
 // An OrderItem slice to hold data from returned rows.
 var oitems []model.OrderItem

 // Loop through rows, using Scan to assign column data to struct fields.
 for rows.Next() {
  var item model.OrderItem
  if err := rows.Scan(&item.ProductCode, &item.Name, &item.UnitPrice,
   &item.Quantity); err != nil {
   return oitems, err
  }
  oitems = append(oitems, item)
 }
 if err = rows.Err(); err != nil {
  return oitems, err
 }
 return oitems, nil
}

By embedding WithConfig[T], the Service Weaver runtime can take per-component configuration information found in the application config file and use it to initialize the contents of T. In the preceding code block, config struct is used for the implementation for generic type T.

Here’s the implementation for config struct:

Listing 4. config struct to be used with WithConfig[T]

type config struct {
 Driver string //`toml:"Driver"` -> Name of the database driver.
 Source string //`toml:"Source"` -> Database server source URI.
}

func (cfg *config) Validate() error {
 if len(cfg.Driver) == 0 {
  return errors.New("DB driver is not provided")
 }
 if len(cfg.Source) == 0 {
  return errors.New("DB source is not provided")
 }
 return nil
}

Because we have embedded weaver.WithConfig[config] into our component, the component initialization will automatically take the component information found in the application config file.

Because we have implemented persistence as a Service Weaver component, we can make it as a reference to other Service Weaver components as shown below:

Listing 5. Make reference to persistence component along with other components

type Server struct {
 weaver.Implements[weaver.Main]

 handler http.Handler // http router instance

 paymentService      weaver.Ref[paymentservice.Service]
 notificationService weaver.Ref[notificationservice.Service]
 orderRepository     weaver.Ref[cockroachdb.Repository]

 orderapi weaver.Listener //`weaver:"orderapi"`
}

Finally, making persistence with storage, is simply access the referenced component and call the corresponding methods as shown below:

Listing 6. Calling the method of persistence component

// persistence using orderRepository component
 if err := s.orderRepository.Get().CreateOrder(ctx, order); err != nil {
 }

The source code of the example demo is available on github at: https://github.com/shijuvar/service-weaver/tree/main/orderapp.

You can follow me on LinkedIn. In India, I do provide architecture consulting and training for building distributed systems in the Go programming language. My distributed systems programming Masterclass, includes the guidance for building distributed applications with Service Weaver.