Persisting Objects with Voyage

Johan Fabry and Esteban Lorenzano with Damien Cassou and Norbert Hartl

Voyage is a small persistence framework developed by Esteban Lorenzano, constructed as a small layer between the objects and a persistency mechanism. It is purely object-oriented and has as a goal to present a minimal API to most common development usages. Voyage is a common layer for different backends but currently it supports just two: an in-memory layer and a backend for the MongoDB database (

The in-memory layer is useful to prototype applications quickly and for initial development without a database back-end, for example using the Smalltalk image as the persistency mechanism.

The MongoDB database backend stores the objects in a document-oriented database. In MongoDB each stored entity is a JSON-style document. This document-centric nature allows for persisting complex object models in a fairly straightforward fashion. MongoDB is not an object database, like Gemstone, Magma or Omnibase, so there still is a small gap to be bridged between objects and documents. To bridge this gap, Voyage contains a mapper converting objects to and from documents. This mapper is equivalent to an Object-Relational Mapper (ORM) when using relational databases. While this mapper does not solve all the known impedance mismatch issues when going from objects to a database, we find that using a document database fits better with the object world than a combination of a ORM and a relational database. This is because document databases tend to provide better support for the dynamic nature of the object world.

Voyage provides a default way in which objects are stored in the database. Fine-grained configuration of this can be performed using Magritte descriptions. Voyage also includes a query API, which allows specific objects to be retrieved from a MongoDB database. We will discuss each of these features in this text.

This text started as a number of blog posts by Esteban Lorenzano, which have been extensively reworked by Johan Fabry, and including additional information shared by Sabine Manaa and Norbert Hartl.

1. Setup

1.1. Load Voyage

To install Voyage, including support for the MongoDB database, go to the Configurations Browser (in the World Menu/Tools) and load ConfigurationOfVoyageMongo. Or alternatively execute in a workspace:

Gofer it
   url: '';
   configurationOf: 'VoyageMongo';

This will load all that is needed to persist objects into a Mongo database.

1.2. Install MongoDB

Next is to install the MongoDB database. How to do this depends on the operating system, and is outside of the scope of this text. We refer to the MongoDB website for more information.

1.3. Create A repository

In Voyage, all persistent objects are stored in a repository. The kind of repository that is used determines the storage backend for the objects.

To use the in-memory layer for Voyage, an instance of VOMemoryRepository needs to be created, as follows:

repository := VOMemoryRepository new

In this text, we shall however use the MongoDB backend. To start a new MongoDB repository or connect to an existing repository create an instance of VOMongoRepository, giving as parameters the hostname and database name. For example, to connect to the database databaseName on the host mongo.db.url execute the following code:

repository := VOMongoRepository
   host: 'mongo.db.url'
   database: 'databaseName'.

Alternatively, using the message host:port:database: allows to specify the port to connect to. Lastly, if authentication is required, this can be done using the message host:database:username:password: or the message host:port:database:username:password:.

1.4. Singleton Mode and Instance Mode

Voyage can work in two different modes:

  • Singleton mode: There is an unique repository in the image, which works as a singleton keeping all the data. When you use this mode, you can program using a "behavioral complete" approach where instances respond to a certain vocabulary (see below for more details about vocabulary and usage).
  • Instance mode: You can have an undetermined number of repositories living in the image. Of course, this mode requires you to make explicit which repositories you are going to use.

By default, Voyage works in instance mode: the returned instance has to be passed as an argument to all database API operations. Instead of having to keep this instance around, a convenient alternative is to use Singleton mode. Singleton mode removes the need to pass the repository as an argument to all database operations. To use Singleton mode, execute:

repository enableSingleton.

Only one repository can be the singleton, hence executing this line will remove any other existing repositories from Singleton mode! In this document, we cover Voyage in Singleton mode, but using it in Instance mode is straightforward as well. See the protocol persistence of VORepository for more information.

1.5. Voyage API

The following two tables show a representative subset of the API of Voyage. These methods are defined on Object and Class, but will only truly perform work if (instances of) the receiver of the message is a Voyage root. See the voyage-model-core-extensions persistence protocol on both classes for the full API of Voyage.

First we show Singleton mode:

save stores an object into repository (insert or update)
remove removes an object from repository
removeAll removes all objects of class from repository
selectAll retrieves all objects of some kind
selectOne: retrieves first object that matches the argument
selectMany: retrieves all objects that matches the argument

Second is Instance mode. In Instance mode, the first argument is always the repository on which to perform the operation.

save: stores an object into repository (insert or update)
remove: removes an object from repository
removeAll: removes all objects of class from repository
selectAll: retrieves all objects of some kind
selectOne:where: retrieves first object that matches the where clause
selectMany:where: retrieves all objects that matches the where clause

1.6. Resetting or Dropping the Database Connection

In a deployed application, there should be no need to close or reset the connection to the database. Also, Voyage re-establishes the connection when the image is closed and later reopened.

However, when developing, resetting the connection to the database may be needed to reflect changes. This is foremost required when changing storage options of the database (see section 3). Performing a reset is achieved as follows:

VORepository current reset.

In case the connection to the database needs to be dropped, this is performed as follows:

VORepository setRepository: nil.

2. Storing Objects

To store objects, the class of the object needs to be declared as being a root of the repository. All repository roots are points of entry to the database. Voyage stores more than just objects that contain literals. Complete trees of objects can be stored with Voyage as well, and this is done transparently. In other words, there is no need for a special treatment to store trees of objects. However, when a graph of objects is stored, care must be taken to break loops. In this section we discuss such basic storage of objects, and in section 3 on Enhancing Storage we show how to enhance and/or modify the way objects are persisted.

2.1. Basic Storage

Let's say we want to store an Association (i.e. a pair of objects). To do this, we need to declare that the class Association is storable as a root of our repository. To express this we define the class method isVoyageRoot to return true.

Association class>>isVoyageRoot
   ^ true

We can also define the name of the collection that will be used to store documents with the voyageCollectionName class method. By default, Voyage creates a MongoDB collection for each root class with name the name of the class.

Association class>>voyageCollectionName
   ^ 'Associations'

Then, to save an association, we need to just send it the save message:

anAssociation := #answer->42.
anAssociation save.

This will generate a collection in the database containing a document of the following structure:

   "_id" : ObjectId("a05feb630000000000000000"),
   "#instanceOf" : "Association",
   "#version" : NumberLong("3515916499"),
   "key" : 'answer',
   "value" : 42

The stored data keeps some extra information to allow the object to be correctly reconstructed when loading:

  • instanceOf records the class of the stored instance. This information is important because the collection can contain subclass instances of the Voyage root class.
  • version keeps a marker of the object version that is committed. This property is used internally by Voyage for refreshing cached data in the application. Without a version field, the application would have to refresh the object by frequently querying the database.

Note that the documents generated by Voyage are not directly visible using Voyage itself, as the goal of Voyage is to abstract away from the document structure. To see the actual documents you need to access the database directly. For MongoDB this can be done through Mongo Browser, which is loaded as part of Voyage (World->Tools->Mongo Browser). Other options for MongoDB are to use the mongo command line interface or a GUI tool such as RoboMongo (Multi-Platform) or MongoHub (for Mac).

2.2. Embedding Objects

Objects can be as simple as associations of literals or more complex: objects can contain other objects, leading to a tree of objects. Saving such objects is as simple as sending the save message to them. For example, let's say that we want to store rectangles and that each rectangle contains two points. To achieve this, we specify that the Rectangle class is a document root as follows:

Rectangle class>>isVoyageRoot
   ^ true

This allows rectangles to be saved to the database, for example as shown by this snippet:

aRectangle := 42@1 corner: 10@20.
aRectangle save.

This will add a document to the rectangle collection of the database with this structure:

   "_id" : ObjectId("ef72b5810000000000000000"),
   "#instanceOf" : "Rectangle",
   "#version" : NumberLong("2460645040"),
   "origin" : {
      "#instanceOf" : "Point",
      "x" : 42,
      "y" : 1
   "corner" : {
      "#instanceOf" : "Point",
      "x" : 10,
      "y" : 20

2.3. Referencing other Roots

Sometimes the objects are trees that contain other root objects. For instance, you could want to keep users and roles as roots, i.e. in different collections, and a user has a collection of roles. If the embedded objects (the roles) are root objects, Voyage will store references to these objects instead of including them in the document.

Returning to our rectangle example, let's suppose we want to keep the points in a separate collection. In other words, now the points will be referenced instead of embedded.

After we add isVoyageRoot to Point class, and save the rectangle, in the rectangle collection, we get the following document:

   "_id" : ObjectId("7c5e772b0000000000000000"),
   "#instanceOf" : "Rectangle",
   "#version" : 423858205,
   "origin" : {
      "#collection" : "point",
      "#instanceOf" : "Point",
      "__id" : ObjectId("7804c56c0000000000000000")
   "corner" : {
      "#collection" : "point",
      "#instanceOf" : "Point",
      "__id" : ObjectId("2a731f310000000000000000")

In addition to this, in the collection point we also get the two following entities:

   "_id" : ObjectId("7804c56c0000000000000000"),
   "#version" : NumberLong("4212049275"),
   "#instanceOf" : "Point",
   "x" : 42,
   "y" : 1

   "_id" : ObjectId("2a731f310000000000000000"),
   "#version" : 821387165,
   "#instanceOf" : "Point",
   "x" : 10,
   "y" : 20

2.4. Breaking Cycles in Graphs

When the objects to be stored contain a graph of embedded objects instead of a tree, i.e. when there are cycles in the references that the embedded objects have between them, the cycles between these embedded objects must be broken. If not, storing the objects will cause an infinite loop. The most straightforward solution is to declare one of the objects causing the cycle as a Voyage root. This effectively breaks the cycle at storage time, avoiding the infinite loop.

For example, in the rectangle example say we have a label inside the rectangle, and this label contains a piece of text. The text also keeps a reference to the label in which it is contained. In other words there is a cycle of references between the label and the text. This cycle must be broken in order to persist the rectangle. To do this, either the label or the text must be declared as a Voyage root.

An alternative solution to break cycles, avoiding the declaration of new voyage roots, is to declare some fields of objects as transient and define how the graph must be reconstructed at load time. This will be discussed in the following section.

2.5. Storing Instances of Date in Mongo

A known issue of mongo is that it does not make a difference between Date and DateAndTime, so even if you store a Date instance, you will retrieve a DateAndTime instance. You will have to transform it back to Date manually when materializing the object.

3. Enhancing Storage

How objects are stored can be changed by adding Magritte descriptions to their classes. In this section, we first talk about configuration options for the storage format of the objects. Then we treat more advanced concepts such as loading and saving of attributes, which can be used, for example, to break cycles in embedded objects.

3.1. Configuring Storage

Consider that, continuing with the rectangle example but using embedded points, we add the following storage requirements:

  • We need to use a different collection named rectanglesForTest instead of rectangle.
  • We only store instances of the Rectangle class in this collection, and therefore the instanceOf information is redundant.
  • The origin and corner attributes are always going to be points, so the instanceOf information there is redundant as well.

To implement this, we use Magritte descriptions with specific pragmas to declare properties of a class and to describe both the origin and corner attributes.

The method mongoContainer is defined as follows: First it uses the pragma <mongoContainer> to state that it describes the container to be used for this class. Second it returns a specific VOMongoContainer instance. This instance is configured such that it uses the rectanglesForTest collection in the database, and that it will only store Rectangle instances.

Note that it is not required to specify both configuration lines. It is equally valid to only declare that the collection to be used is rectanglesForTest, or only specify that the collection contains just Rectangle instances.

Rectangle class>>mongoContainer

   ^ VOMongoContainer new
      collectionName: 'rectanglesForTest';
      kind: Rectangle;

The two other methods use the pragma <mongoDescription> and return a Mongo description that is configured with their respective attribute name and kind, as follows:

Rectangle class>>mongoOrigin

   ^ VOMongoToOneDescription new
      attributeName: 'origin';
      kind: Point;
Rectangle class>>mongoCorner

   ^ VOMongoToOneDescription new
      attributeName: 'corner';
      kind: Point;

After resetting the repository with:

VORepository current reset

a saved rectangle, now in the rectanglesForTest collection, will look more or less as follows:

   "_id" : ObjectId("ef72b5810000000000000000"),
   "#version" : NumberLong("2460645040"),
   "origin" : {
      "x" : 42,
      "y" : 1
   "corner" : {
      "x" : 10,
      "y" : 20

Other configuration options for attribute descriptions are:

  • beEager declares that the referenced instance is to be loaded eagerly (the default is lazy).
  • beLazy declares that referenced instances are loaded lazily.
  • convertNullTo: when retrieving an object whose value is Null (nil), instead return the result of evaluating the block passed as argument.

For attributes which are collections, the VOMongoToManyDescription needs to be returned instead of the VOMongoToOneDescription. All the above configuration options remain valid, and the kind: configuration option is used to specify the kind of values the collection contains.

VOMongoToManyDescription provides a number of extra configuration options:

  • kindCollection: specifies the class of the collection that is contained in the attribute.
  • convertNullToEmpty when retrieving a collection whose value is Null (nil), it returns an empty collection.

3.2. Custom Loading and Saving of Attributes

It is possible to write specific logic for transforming attributes of an object when written to the database, as well as when read from the database. This can be used, e.g., to break cycles in the object graph without needing to declare extra Voyage roots. To declare such custom logic, a MAPluggableAccessor needs to be defined that contains Smalltalk blocks for reading the attribute from the object and writing it to the object. Note that the names of these accessors can be counter-intuitive: the read: accessor defines the value that will be stored in the database, and the write: accessor defines the transformation of this retrieved value to what is placed in the object. This is because the accessors are used by the Object-Document mapper when reading the object to store it to the database and when writing the object to memory, based on the values obtained from the database.

Defining accessors allows, for example, a Currency object that is contained in an Amount to be written to the database as its' three letter abbreviation (EUR, USD, CLP, ...). When loading this representation, it needs to be converted back into a Currency object, e.g. by instantiating a new Currency object. This is achieved as follows:

Amount class>>mongoCurrency

   ^ VOMongoToOneDescription new
      attributeName: 'currency';
      accessor: (MAPluggableAccessor
         read: [ :amount | amount currency abbreviation ]
         write: [ :amount :value | amount currency: (Currency fromAbbreviation: value) ]);

Also, a post-load action can be defined for an attribute or for the containing object, by adding a postLoad: action to the attribute descriptor or the container descriptor. This action is a one-parameter block, and will be executed after the object has been loaded into memory with as argument the object that was loaded.

Lastly, attributes can be excluded from storage (and hence retrieval) by returning a VOMongoTransientDescription instance as the attribute descriptor. This allows to place cut-off points in the graph of objects that is being saved, i.e. when an object contains a reference to data that should not be persisted in the database. This may also be used to break cycles in the stored object graph. It however entails that when retrieving the graph from the database, attributes that contain these objects will be set to nil. To address this, a post-load action can be specified for the attribute descriptor or the container descriptor, to set these attributes to the correct values.

3.3. A few Words Concerning the OID

The mongo ObjectId (OID) is a unique field acting as a primary key. It is a 12-byte BSON type, constructed using:

  • a 4-byte value representing seconds passed since the Unix epoch,
  • a 3-byte machine identifier,
  • a 2-byte process id,
  • a 3-byte counter, starting with a random value.

Objects which are added into a mongo root collection get a unique id, instance of OID. If you create such an object and then ask it for its OID by sending it voyageId, you get the OID. The instance variable value of the OID contains a LargePositiveInteger that corresponds to the mongo ObjectId.

It is possible to create and use your own implementation of OIDs and put these objects into the mongo database. But this is not recommended as you possibly may no longer be able to query these objects by their OID (by using voyageId), since mongo expects a certain format. If you do, you should check your format by querying for it in the mongo console, for example as below. If you get the result Error: invalid object id: length, then you will not be able to query this object by id.

> db.Trips.find({"person.__id" : ObjectId("190372")})
Fri Aug 28 14:21:10.815 Error: invalid object id: length

An extra advantage of the OID in the mongo format is that these are ordered by creation date and time and as a result you have an indexed "creationDateAndTime" attribute for free (since there is a non deletable index on the field of the OID _id).

4. Querying in Voyage

Voyage allows to selectively retrieve object instances though queries on the database. When using the in-memory layer, queries are standard Smalltalk blocks. When using the MongoDB back-end, the MongoDB query language is used to perform the searches. To specify these queries, MongoDB uses JSON structures, and when using Voyage there are two ways in which these can be constructed. MongoDB queries can be written either as blocks or as dictionaries, depending on their complexity. In this section, we first discuss both ways in which queries can be created, and we end the section by talking about how to execute these queries.

4.1. Basic Object Retrieval using Blocks or MongoQueries

The most straightforward way to query the database is by using blocks when using the in-memory layer or MongoQueries when using the MongoDB back-end. In this discussion we will focus on the use of MongoQueries, as the use of blocks is standard Smalltalk.

MongoQueries is not part of Voyage itself but part of the MongoTalk layer that Voyage uses to talk to MongoDB. MongoTalk was made by Nicolas Petton and provides all the low-level operations for accessing MongoDB. MongoQueries transforms, within certain restrictions, regular Pharo blocks into JSON queries that comply to the form that is expected by the database. In essence, MongoQueries is an embedded Domain Specific Language to create MongoDB queries. Using MongoQueries, a query looks like a normal Pharo expression (but the language is much more restricted than plain Smalltalk).

Using MongoQueries, the following operators may be used in a query:

< <= > >= = ~= Regular comparison operators
& AND operator
| OR operator
not NOT operator
at: Access an embedded document
where: Execute a Javascript query

For example, a query that selects all elements in the database whose name is John is the following:

[ :each | each name = 'John' ]

A slightly more complicated query is to find all elements in the database whose name is John and the value in orders is greater than 10.

[ :each | (each name = 'John') & (each orders > 10 ) ]

Note that this way of querying only works for querying values of the object but not values of references to other objects. For such case you should build your query using ids, as traditionally done in relational database, which we talk about next. However the best solution in the Mongo spirit of things is to revisit the object model to avoid relationships that are expressed with foreign keys.

4.1.1. Quering with Elements from another Root Document

With No-SQL databases, it is impossible to query on multiple collections (the equivalent of a JOIN statement in SQL). You have two options: alter your schema, as suggested above, or write application-level code to reproduce the JOIN behavior. The latter option can be done by sending the voyageId message to an object already returned by a previous query and using that id to match another object. An example where we match colors color to a reference color refCol is as follows:

[ :each | (each at: 'color.__id') = refCol voyageId ]
4.1.2. Using the at: Message to Access Embedded Documents

Since MongoDB stores documents of any complexity, it is common that one document is composed of several embedded documents, for example:

   "origin" : {
      "x" : 42,
      "y" : 1
   "corner" : {
      "x" : 10,
      "y" : 20

In this case, to search for objects by one of the embedded document elements, the message at:, and the field separator "." needs to be used. For example, to select all the rectangles whose origin x value is equal to 42, the query is as as follows.

[ :each | (each at: 'origin.x') = 42 ]
4.1.3. Using the where: Message to Perform Javascript Comparisons

To perform queries which are outside the capabilities of MongoQueries or even the MongoDB query language, MongoDB provides a way to write queries directly in Javascript using the $where operand. This is also possible in MongoQueries by sending the where: message:

In the following example we repeat the previous query with a Javascript expression:

[ :each | each where: 'this.origin.x == 42' ].

More complete documentation about the use of $where is in the MongoDB where documentation.

4.2. Using JSON Queries

When MongoQueries is not powerful enough to express your query, you can use a JSON query instead. JSON queries are the MongoDB query internal representation, and can be created straightforwardly in Voyage. In a nutshell: a JSON structure is mapped to a dictionary with pairs. In these pairs the key is a string and the value can be a primitive value, a collection or another JSON structure (i.e., another dictionary). To create a query, we simply need to create a dictionary that satisfies these requirements.

The use of JSON queries is strictly for when using the MongoDB back-end. Other back-ends, e.g., the in-memory layer, do not provide support for the use of JSON queries.

For example, the first example of the use of MongoQueries is written as a dictionary as follows:

{ 'name' -> 'John' } asDictionary

Dictionary pairs are composed with AND semantics. Selecting the elements having John as name AND whose orders value is greater than 10 can be written like this:

   'name' -> 'John'.
   'orders' -> { '$gt' : 10 } asDictionary
} asDictionary

To construct the "greater than" statement, a new dictionary needs to be created that uses the MongoDB $gt query selector to express the greater than relation. For the list of available query selectors we refer to the MongoDB Query Selectors documentation.

4.2.1. Querying for an Object by OID

If you know the ObjectId for a document, you can create an OID instance with this value and query for it.

{('_id' -> (OID value: 16r55CDD2B6E9A87A520F000001))} asDictionary.

Note that both of the following are equivalent:

OID value: 26555050698940995562836590593. "dec"
OID value: 16r55CDD2B6E9A87A520F000001. "hex"

If you have an instance which is in a root collection, then you can ask it for its voyageId and use that ObjectId in your query.

4.2.2. Using dot Notation to Access Embedded Documents

To access values embedded in documents with JSON queries, the dot notation is used. For example, the query representing rectangles whose origin have 42 as their x values can be expressed this way:

   'origin.x' -> {'$eq' : 42} asDictionary
} asDictionary
4.2.3. Expressing OR Conditions in the Query

To express an OR condition, a dictionary whose key is '$or' and whose values are the expression of the condition is needed. The following example shows how to select all objects whose name is John that have more than ten orders OR objects whose name is not John and has ten or less orders:

{ '$or' :
         'name' -> 'John'.
         'orders' -> { '$gt': 10 } asDictionary
      } asDictionary.
         'name' -> { '$ne': 'John'} asDictionary.
         'orders' -> { '$lte': 10 } asDictionary
      } asDictionary.
} asDictionary.
4.2.4. Going Beyond MongoQueries Features

Using JSON queries allows to use features that are not present in MongoQueries, for example the use of regular expressions. Below is a query that searches for all documents with a fullname.lastName that starts with the letter D:

   'fullname.lastName' -> {
      '$regexp': '^D.*'.
      '$options': 'i'.
   } asDictionary.
} asDictionary.

The option i for a regular expression means case insensitivity. More options are described in the documentation of the $regex operator.

This example only briefly illustrates the power of JSON queries. Many more different queries can be constructed, and the complete list of operators and usages is in the MongoDB operator documentation

4.3. Executing a Query

Voyage has a group of methods to perform searches. To illustrate the use of these methods we will use the stored Point example we have presented before. Note that all queries in this section can be written either as MongoQueries or as JSON queries, unless otherwise specified.

4.3.1. Basic Object Retrieval

The following methods provide basic object retrieval.

  • selectAll Retrieves all documents in the corresponding database collection. For example, Point selectAll will return all Points.
  • selectOne: Retrieves one document matching the query. This maps to a detect: method and takes as argument a query specification (either a MongoQuery or a JSON Query). For example, Point selectOne: [:each | each x = 42] or alternatively Point selectOne: { 'x' -> 42 } asDictionary.
  • selectMany: Retrieves all the documents matching the query. This maps to a select: method and takes as argument a query specification, like above.
4.3.2. Limiting Object Retrieval and Sorting

The methods that query the database look similar to their equivalent in the Collection hierarchy. However unlike regular collections which can operate fully on memory, often Voyage collection queries need to be customized in order to optimize memory consumption and/or access speed. This is because there can be literally millions of documents in each collection, surpassing the memory limit of Pharo, and also the database searches have a much higher performance than the equivalent code in Pharo.

The first refinement to the queries consist in limiting the amount of results that are returned. Of the collection of all the documents that match, a subset is returned that starts at the index that is given as argument. This can be used to only retrieve the first N matches to a query, or go over the query results in smaller blocks, as will be shown next in the simple paginator example.

  • selectMany:limit: Retrieves a collection of objects from the database that match the query, up to the given limit. An example of this is Point selectMany: [:each | each x = 42] limit: 10
  • selectMany:limit:offset: Retrieves a collection of objects from the database that match the query. The first object retrieved will be at the offset position plus one of the results of the query, and up to limit objects will be returned. For example, if the above example matched 25 points, the last 15 points will be returned by the query Point selectMany: [:each | each x = 42] limit: 20 offset: 10 (any limit argument greater than 15 will do for this example).

The second customization that can be performed is to sort the results. To use this, the class VOOrder provides constants to specify ascending or descending sort order.

  • selectAllSortBy: Retrieves all documents, sorted by the specification in the argument, which needs to be a JSON query. For example, Point selectAllSortBy: { #x -> VOOrder ascending} asDictionary returns the points in ascending x order.
  • selectMany:sortBy: Retrieves all the documents matching the query and sorts them. For example to return the points where x is 42, in descending y order: Point selectMany: { 'x' -> 42 } asDictionary sortBy: { #y -> VOOrder descending } asDictionary.
  • selectMany:sortBy:limit:offset: Provides for specifying a limit and offset to the above query.
4.3.3. A Simple Paginator Example

Often you want to display just a range of objects that belong to the collection, e.g. the first 25, or from 25 to 50, and so on. Here we present a simple paginator that implements this behavior, using the selectMany:limit:offset: method.

First we create a class named Paginator. To instantiate it, a Voyage root (aClass) and a query (aCondition) need to be given.

Object subclass: #Paginator
   instanceVariableNames: 'collectionClass where pageCount'
   classVariableNames: ''
   package: 'DemoPaginator'

Paginator class>>on: aClass where: aCondition
   ^ self basicNew
      initializeOn: aClass where: aCondition

Paginator>>initializeOn: aClass where: aCondition
   self initialize.
   collectionClass := aClass.
   where := aCondition

Then we define the arithmetic to get the number of pages for a page size and a given number of entities.

   ^ 25

   ^ pageCount ifNil: [ pageCount := self calculatePageCount ]

   | count pages |
   count := self collectionClass count: self where.
   pages := count / self pageSize.
   count \\ self pageSize > 0
      ifTrue: [ pages := pages + 1].
   ^ count

The query that retrieves only the elements for a given page is then implemented as follows:

Paginator>>page: aNumber
   ^ self collectionClass
      selectMany: self where
      limit: self pageSize
      offset: (aNumber - 1) * self pageSize

5. Creating and Removing Indexes

There are a number of useful features in MongoDB that are not present in Voyage but still can be performed from within Pharo, the most important one being the management of indexes.

5.1. Creating Indexes by using OSProcess

It is not yet possible to create and remove indexes from Voyage, but this can nonetheless be done by using OSProcess.

For example, assume there is a database named myDB with a collection named Trips. The trips have an embedded collection with receipts. The receipts have an attribute named description. The following creates an index on description:

OSProcess command:
   '/{pathToMongoDB}/MongoDB/bin/mongo --eval ',

Removing all indexes on the Trips collection can be done as follows:

OSProcess command:
   '/{pathToMongoDB}/MongoDB/bin/mongo --eval ',

5.2. Verifying the use of an Index

To ensure that a query indeed uses the index, ".explain()" can be used in the mongo console. For example, if we add the index on description as above, run a query and add .explain() we see, that only a subset of documents were scanned.

> db.Trips.find({"receipts.description":"a"})
   "cursor" : "BtreeCursor receipts.receiptDescription_1",
   "isMultiKey" : true,
   "n" : 2,
   "nscannedObjects" : 2,
   "nscanned" : 2,
   "nscannedObjectsAllPlans" : 2,
   "nscannedAllPlans" : 2,


After removing the index, all documents are scanned (in this example there are 246):

> db.Trips.find({"receipts.description":"a"}
   "cursor" : "BasicCursor",
   "isMultiKey" : false,
   "n" : 2,
   "nscannedObjects" : 246,
   "nscanned" : 246,
   "nscannedObjectsAllPlans" : 246,
   "nscannedAllPlans" : 246,


6. Conclusion

In this chapter we presented Voyage, a persistence programming framework. The strength of Voyage lies in the presence of the object-document mapper and MongoDB back-end. We have shown how to store objects in, and remove object from the database, and how to optimise the storage format. This was followed by a discussion of querying the database; showing the two ways in which queries can be constructed and detailing how queries are ran. We ended this chapter by presenting how we can construct indexes in MongoDB databases, even though Voyage does not provide direct support for it.