Dynamit Specification

Dynamit Specification Package version: 0.1.23+ Date: 2017-11-19

Author: Erik von Krusenstierna, [email protected]

Table of Contents 1. Introduction.................................................................................................................................... 3 1.1. Static vs. dynamic .................................................................................................................. 3 1.2. How does it work? .................................................................................................................. 3 1.3. Distribution ............................................................................................................................. 4 2. Using Dynamit ............................................................................................................................... 5 2.1. A simple example ................................................................................................................... 5 2.2. Dynamic table declarations ................................................................................................... 6 2.3. DynamitConfig ........................................................................................................................ 6 2.4. Dynamit.Finder ....................................................................................................................... 7 2.5. Dynamit.DValue ...................................................................................................................... 8 3. Casts, JSON and data types ......................................................................................................... 9 4. C# Dynamic .................................................................................................................................. 10 5. See also ........................................................................................................................................ 10

2

(c) 2017 Mopedo AB. All rights reserved.

1. Introduction Dynamit is a Starcounter class library and .NET package developed by Mopedo, for creating and querying dynamic database tables in Starcounter applications. This article will cover the basics of Dynamit and how to set it up in a Visual Studio project. Dynamit is distributed as a package on the NuGet Gallery, and the easiest way to install it in an active Visual Studio project is by entering the following into the NuGet Package Manager console: PM> Install-Package Dynamit

See also: https://www.nuget.org/packages/Dynamit

If you have any questions or comments about this document, please contact [email protected]

1.1. Static vs. dynamic To illustrate the difference between dynamic and non-dynamic – static – tables, consider the objectoriented programming concept object, and its standard implementations in Java and JavaScript respectively. In Java – which is class-based – objects are static with regards to their members, and this structure is defined in compile-time in a class definition. We can change the contents of objects – that is, assign values to instance fields – during runtime, but we cannot change the structure of objects without first recompiling the application. In JavaScript – which is prototype-based – however, both the content and the structure of objects can change during runtime. As far as .NET developers working with Starcounter are concerned, rows in static Starcounter tables behave like objects in Java. We can create new rows in runtime, but the table schema is defined during compile-time. Rows in dynamic tables, however, behave like JavaScript objects. There is no common schema of property names and types that applies to all rows in a dynamic table, and their members can have any name and data type, regardless of members and types of other entities in the same table.

1.2. How does it work? Dynamit achieves dynamic Starcounter tables by means of normalization. All data types used for values are ordered in value object classes, and are assigned their own Starcounter database tables. A row in these tables can, for example, be the string "Foo" or the integer 42. Since Starcounter makes no distinction between database objects and table rows, the term “value object” is used to refer to these table rows. Value objects are then assigned to key-value pairs with Starcounter object references. These key-value pairs contain a string key, a reference to a value object and a stored value object hash code. The key-value pair is dynamically typed, since its value object can be of any value object class. The dynamic nature of how key-value pairs reference the values of their value objects means that we cannot index or run queries against the actual values, which is a limitation we need to be aware of. We can, however, index and query the value hash codes, which is how querying dynamic tables can still be fast (for most queries).

3

(c) 2017 Mopedo AB. All rights reserved.

The dynamic key-value pairs rows then contain many-to-one references to dictionary objects (also with Starcounter object references) that are essentially lists of key-value pairs. Consider the following JSON object: { "Label": "Double Espresso", "Product_ID": 42, "Product_date": "2017-01-05T00:00:00", "Price": 3.25, "Group": "A1" }

If this object was to be stored in a dynamic table, it would result in five key-value pairs – each with a key corresponding to the property name, a value hash corresponding to the hash code of the value, and a reference to a value object of dynamic type, containing the actual value. There would also be a dictionary object, and all five key-value pairs would contain a reference to that dictionary.

1.3. Distribution Dynamit is currently free to use, and is distributed over the NuGet package manager. If you have any questions, comments or bug reports, or if you want to contribute to the Dynamit project, please contact [email protected].

4

(c) 2017 Mopedo AB. All rights reserved.

2. Using Dynamit 2.1. A simple example using Dynamit; public class App { public static void Main() { DynamitConfig.Init();

// A call to Init() initializes Dynamit

Db.Transact(() => { new Product

// This is our dynamic table row

{ ["Label"] = "Double Espresso", ["Product_ID"] = 42, ["Product_date"] = new DateTime(2017, 01, 05), ["Price"] = 3.25, ["Group"] = "A1" }; }); } }

var product = Finder.All.Where(o => o["Product_ID"] == 42); product["Price"] // 3.25 product["Label"].Length // 15 product["Product_date"].AddDays(2).ToString("O") // "2017-01-07T00:00:00.0000000" product is IDictionary // true Finder.All.Where(row => row["Product_ID"] > 10).First()["Label"] // "Double Espresso"

5

(c) 2017 Mopedo AB. All rights reserved.

2.2. Dynamic table declarations All dynamic tables inherit from the public abstract class Dynamit.DDictionary, which in turn implements IDictionary. In the example above, Product is a subclass of DDictionary. Product also have a separate table that contains all its key-value pairs. This table is declared by creating a subclass of the public abstract class Dynamit.DKeyValuePair.

Below is the complete dynamic table declaration for Product. The structure is the same for all dynamic table declarations. A key-value pair class is declared, that inherits from DKeyValuePair, a dictionary class is declared that inherits from DDictionary, and the dictionary class also implements the generic IDDictionary interface, which Dynamit then uses to connect the key-value pair class with the dictionary class under the hood. The dictionary class also needs to implement the abstract method NewKeyPair, which returns a new key-value pair instance. We also declare a constructor in the key-value pair class that calls the base constructor. public class Product : DDictionary, IDDictionary { public ProductKVP NewKeyPair(Product dict, string key, object value = null) { return new ProductKVP(dict, key, value); } } public class ProductKVP : DKeyValuePair { public ProductKVP(DDictionary dict, string key, object value = null) : base(dict, key, value) { } }

Q: Why do we need to declare our own sub-classes of DDictionary and DKeyValuePair? Why not just have these classes be concrete classes, and work with them directly? A: The reason for this is that we want to have as many separate tables as possible, to speed up queries if we have many dynamic tables. We also want to expose the constructor for the key-value pairs, so that we can add additional logic that checks keys and values before they are inserted.

2.3. DynamitConfig To enable fast queries against dynamic tables, we need certain indexes to be registered in Starcounter. The method DynamitConfig.Init will set up the necessary indexes during app startup, which is recommended if you do not set up these indexes yourself. By calling DynamitConfig.Init on app startup, the following is done: 1. All dynamic table declarations are checked for declaration errors, and exceptions are thrown if there are errors. 2. Indexes are set up for all key-value pair tables on columns: a. Dictionary 6

(c) 2017 Mopedo AB. All rights reserved.

b. Dictionary, Key c. Key, ValueHash By setting the parameters of the call to DynamitConfig.Init, you can skip step 2. In the call to Init you can also set whether escaped strings such as “\"Example\"” should be understood as “Example” when setting values in dynamic tables. This is necessary to be able to unambiguously identify values that should be parsed as strings rather than datetimes when using other solutions that write strings to Dynamit tables, for example RESTar.

2.4. Dynamit.Finder Querying dynamic tables using SQL is a bit trickier than querying regular tables. The Finder class gives access to some useful static methods for finding DDictionary entities. class Finder where T : DDictionary, IDDictionary { IEnumerable All; IEnumerable Where(string key, Operator op, dynamic value); IEnumerable Where(params ValueTuple[] equalityConditions); T First(params ValueTuple[] equalityConditions); } // Examples: using static Dynamit.Operator; Finder .Where("Group", EQUALS, "A1")

// C#7 ValueTuple literal

.Where(x => x.SafeGet("Price") > 3);

// regular LINQ

Finder.All returns all entities of type T.

To make queries as fast as possible, it is always recommended to use the Finder.Where methods (as opposed to Finder.All + LINQ) whenever the query contains equality conditions, since equality conditions can be evaluated much faster than other conditions due to the way values are hashed in key-value pairs. Consider the two examples below as good practice: // 1. Find all products with a product id larger than 5: Finder.All.Where(d => d["ProductId"] > 5); // Here we must use All + LINQ, since the ‘>’ operator generally cannot be applied // to the value hashes // 2. Find all products with a product group of "A1" and price higher than 3: Finder .Where("Group", EQUALS, "A1") .Where(d => d.SafeGet("Price") > 3); // Here we can use Finder.Where for the equality condition, and LINQ for the

7

(c) 2017 Mopedo AB. All rights reserved.

// non-equality conditions. // Tip: use the DDictionary.SafeGet(object) method to get the value of a dynamic // object property, or null if there is no such property.

2.5. Dynamit.DValue The Dynamit namespace also contains a class DValue that can be used to store values of arbitrary data types in static Starcounter database class definitions. In the following example, we need a dynamic member to hold a value of an arbitrary type. [Database] public class Condition : IEntity { public string PropertyName; public OperatorsEnum Operator; public ConditionValue Value;

// This is our DValue member

public void OnDelete() => Value?.Delete(); } public class ConditionValue : DValue { public ConditionValue(object value) => Value = value; }

Example usage: Db.Transact() => { new Condition { PropertyName = "Name", Operator = OperatorsEnum.Equals, Value = new ConditionValue("Tony Wonder") }; });

8

(c) 2017 Mopedo AB. All rights reserved.

3. Casts, JSON and data types Dynamit dynamic tables can contain values of the following data types: Boolean, Byte, DateTime, Decimal, Double, Int16, Int32, Int64, SByte, Single, String, UInt16, UInt32, UInt64

Since DDictionary implements IDictionary, most proper JSON serializers can deserialize straight to DDictionary. The Jil serializer and Newtonsoft Json.net are two examples of such serializers. When deserializing to DDictionary, the serializer will try to insert dynamic values, that is, objects implementing IDynamicMetaObjectProvider into the DDictionary. Dynamit can handle this, and will automatically try to make the following casts, in order, when deserializing values of unknown dynamic type: 1. Item castable to DateTime => DateTime 2. Item castable to String => String 3. Item castable to Boolean => Boolean 4. Item castable to Int32 => Int32 5. Item castable to Int64 => Int64 6. Else, try cast to Decimal 7. Else, fail

Most serializers also know how to serialize DDictionary entities, since they already know how to serialize IDictionary.

9

(c) 2017 Mopedo AB. All rights reserved.

4. C# Dynamic Since package version version 0.1.7, DDictionary objects have support for C# dynamic member binding, which gives an alternative syntax for object property access. Example: dynamic product = null; Db.TransactAsync(() => { product = new Product(); product.Label = "Double Espresso", product.Product_ID = 42, product.Product_date = new DateTime(2017, 01, 05), product.Price = 3.25, product.Group = "A1" }); product.Label // "Double Espresso" product.Product_date.AddDays(1).ToString() // "2017-01-06 12:00:00 AM"

5. See also Dynamit is used by RESTar, a package developed by Mopedo to help create powerful RESTful web services on the Starcounter system. RESTar is also available on NuGet, and has a separate documentation article that explains it, available at https://goo.gl/cR5lph.

10

(c) 2017 Mopedo AB. All rights reserved.