Saturday, May 17, 2014

Initial clicks on the Usergrid Admin portal

The attempt of this post is to show you some of the areas in the Usergrid admin portal. Mainly to get you going on using Usergrid.
If you still do not have your local instance of "the BaaS framework you run" take a look at my previous post at: http://techviewofalearner.blogspot.com/2014/05/running-usergrid-get-started.html

The admin portal at first glance looks like this: 




As you will see, the portal has an Organization and Application set. In this case the Organization is 'test-organization' and the Application is 'test-app'.
I refuse to let you pass this point without an understanding that concept. So I am going to use a few images Sungju Jin (Usergrid commiter) used to explain the internals.

Usergrid supports multi-tenancy by design. The following diagram will give you an idea of what that really means:

Your organization will have many applications, you can think of the application as a sandbox. 
One organization can have many applications and those applications have the following structure:

And when accessing specific data, yeah that is right, go through the organization to your specific application:


Wait a minute, did I just mention 'collection' and 'entity' without explaining what that is ! hmph ! 
Relax, here it is below: 

A collection is basically like a table in sql, we can put items that we model into them (in a non relational DB mindset of course !). Things like 'Users', 'Books', 'Cars', 'Aliens', 'Ghosts' (like the one behind you, don't look back). 
Important thing to note here is that they were always in plural.
An entity is basically like a row in that table, a collection called 'Users' can have the following entity for example:
{

"uuid" : "<ID HERE>", 
"type" : "user",
"username" : "malaka",
"created": 138901448423
"modified" : 138901448423
"activated" : true

}

There you go, that is the basic structure. You have to make sure that is clear, after all the whole point of using a BaaS is not to focus on the back-end implementation. In Usergrid once you have your collections defined (we will see how in a bit) you can do the CRUD (create-read-update-delete) operations on them.
The table below explains how:


if your collection is items, you access it using  https://api.domain.io/my-org-id/my-app-id/items
Doing a GET request to that path gives you all the items available (returns an array of entities - that bulb that went of when you heard the word entities, that is proof you are settling into Usergrid). 
A POST will create a new entity, etc. as shown in the table above.

Now let's do this in Usergrid ! 

Click on 'App data' and then click on the 'Add collection' button:

Then you will see the following prompt: 



Put the name as 'item' (or anything you want) and press 'create'.  The new collection will show up in the list as you can see below:


As you can see it is now in ... sorry ? ah yeah plural. you are completing my sentences, you are really getting the hang of this. 

Now remember the http GET, POST, PUT etc. above ? So to create a new 'item' I have to do a ?? POST ! exactly. When doing this, we pass the attributes and values in  JSON key-value pairs. As the example below shows the value can be a string like "item 1" or even an int like 123. 


We have been given the option to validate our JSON, so why not use it. Click on 'Validate Json' and make sure your Json is valid. Then click 'run query'. Make sure that POST is selected above. 
This will add your new entity to the items collection. 

You can do a GET on the same to retrieve the item just added: 


To see additional details just click on the 'View Details' and you will see: 


The uuid (unique identifier), created time, modified time and other meta data is already added for you. 
You have now made your first collection and entity on Usergrid ! 


Regards, 
Malaka


More information on: 
Usergrid internals by Sungju Jin (Usergrid commiter) at : https://speakerdeck.com/sungjuly/apache-usergrid-internal

How to contribute to Usergrid by David Johnson (Usergrid commiter) at: http://www.slideshare.net/snoopdave/how-to-contribute-to-apache-usergrid










Tuesday, May 13, 2014

Running Usergrid - Get started




The Usergrid project is now open source and in the Apache incubation phase.

**[updated post available, the link can be found at the bottom after the project descriptions.]**

Setup environment for Usergrid: 
Requirements
JDK 1.6: (http://www.oracle.com/technetwork/java/javase/downloads/index.html)
Maven (http://maven.apache.org/): Download and install Maven using the instructions on the Apache Maven website

There are a few ways you can download the source from https://github.com/usergrid/usergrid/
1) Click on 'Download ZIP' or Clone the source and follow the instructions on the site to build it.

2) Create\Use your Github account to fork the repository into your own.

The second approach is recommended if you want to keep up to date with the latest code and/or want to contribute to the project. 


At a high level this it what the project structure looks like:

If we go into the SDK folder we can see the SDKs (software development kits) available for Usergrid. 


The stack contains the main project. The structure of the stack is as follows:

Here is a brief description of the projects from the readme files in the relevant folders.

Build-tools: Holds key configuration files for build plugins (Contains resources like the cassandra.yaml).

Config: Runtime config resources of Usergrid system are contained here.

Core: This contains the core services for Usergrid system.
Usergrid-core contains the persistence layer and shared utilities for powering the Usergrid service. The services layer is contained in Usergrid-services and exposes a higher-level API that's used by the Usergrid-rest web services tier.

Launcher: Usergrid Launcher is a graphical server launcher.
The launcher is a simple Java app that provides a GUI for running the server as a desktop app.

Mongo emulator: This provides the Mongo emulation layer for the Usergrid system  . This is still experimental and incomplete implementation of Mongo emulation layer so Mongo clients can connect to Usergrid. There are a number of drivers and tools for talking to Mongo DB, which has a data model that is very similar to that of Usergrid. In order to make it easier for people to get started quickly with Usergrid, it's desirable for them to be able to integrate it very quickly into their existing applications.
The goal here is to support the Mongo native wire protocol and to map enough of the query and CRUD operations so that this can happen.

Query-Validator: This test package verifies that the Usergrid REST API returns results and does paging and limits in a style similar to that of SqlLite.

Rest: This contains the REST web services for Usergrid system (Usergrid REST API Web App).
This installs as a web-app in Tomcat and has not been extensively tested in other web containers.

Services: This contains the Service layer for Usergrid system.
This is where most of the high-level functionality for accessing Usergrid data is performed. It functions as the glue between the REST API as exposed by Jersey and the entity management functionality in Usergrid-core. This is also where security and account management functionality is contained.
The security model of Usergrid is implemented with Apache Shiro: http://shiro.apache.org/
The services model for Usergrid involves converting the REST web service requests into collection queries and routing these to a set of collection handlers.    This provides a "virtual collection" layer on top of the actual collections that are exposed in the persistence tier.  This makes it possible to:
- implement logic behaviour on top of persistent objects
- enforce object and property visibility

Test-utils: This contains the test utilities for Usergrid system as implied by the name.

Tools: This contains the command line tools for Usergrid system as implied by the name.
The Usergrid Tools are a set of Java programs that can be run from a command-line console.

Websocket: This contains the WebSocket API for Usergrid system. It is still an experimental (incomplete) implementation of Websocket API.  Websockets info can be found at http://en.wikipedia.org/wiki/WebSockets . The Websocket API will allow for asynchronous communications with the Usergrid API. OAuth credentials are put in the query string of the initial GET request since the JavaScript Websocket API provides no mechanism for specifying header contents.
Once a connection is established, there are two mechanisms of usage.
Async Request/Response
The first is to simply access the API the same way as the REST interface provides, except over the open websocket connection. The benefit is there is much less overhead than with the REST API since there is no processing of individual HTTP requests and authentication credentials are checked at the time the websocket is opened as opposed to per-request as is necessary with the stateless REST API.

Subscriptions
The second is to issue subscribe requests against entities and collections and be notified asynchronously as those are updated. Subscriptions against the activity inbox and message queue collections for either users, groups, or the entire application allow connected applications to get real-time updates.
Connecting to an inbox or queue in the websocket URL will automatically start a subscription to that collection. For example, the following websocket URL will listen to a specific user's inbox:
ws://api.usergrid.com:8088/chatapp/users/johndoe/inbox
This is the real-time equivalent of making a regular REST GET request to:
http://api.usergrid.com/chatapp/users/edanuff/inbox
Usergrid listens to websockets on an alternate port than it does standard HTTP requests. Although websockets can coexist on the same ports 80 or 443, Tomcat and other Java servlet containers don't do a particularly great job of handling open async socket connections despite what they claim. For our purposes, we're using a bare metal Netty websocket server that listens on port 8088. This is going to be much more performant that trying to integrate it into Tomcat and allows us to move the websocket servers onto different machines.

Getting started: 

Once you have the source downloaded (downloaded/cloned/forked) you can to the the stack folder and run the Maven command: 

(you need to have your java path, maven path and maven repository setup for the command to be successful)


This will download the required dependencies to the Maven repository which are needed to build the Usergrid project.
Once that is done, we will go into the launcher project (see the description of launcher project above) and run that jar file with the command:


[Since the Usergrid source has been updated please switch to http://malakablogs.blogspot.com/2015/06/running-usergrid-improved-version-with.html and continue reading.]

When this jar file runs, what happens is that an instance of the Grizzly server is made inside the target folder. This is so that users can host Usergrid instantly without the need to set up servers. 
It gets better, users do not even have to setup the Cassendra back-end as an instance of that will be made as well for you as you will see below:
(If however after you have got the hand-on of Usergrid you want to setup your own servers and Cassendra instances refer the documents at: http://usergrid.incubator.apache.org/ )

When the jar file is run you will see the following Graphical User Interface (GUI):


As you can see 'Start Database With Server' is already checked - this creates an instance of the Grizzly server onto which Usergrid will be deployed. 
'Initialize Database on Start' will create an instance of the Cassendra back-end for you to play around as well. (Keep in mind however that unlike a proper instance, this stops when you stop the GUI above).

The 'Console URL' is by default pointer to  http://apigee.github.io/usergrid-portal/. You can go ahead and use that. However later on you can host the 'Portal' web project (see project structure) in your own server and point to that (e.g.: http://localhost/usergrid-portal/). 


   Go ahead and press the Play button. The light will change color from red to orange to green. Green means that the setup was successful and you can now use the system. 



You now have the Usergrid console to play around and learn !  Don't forget to have fun doing it ! 

Regards, 
Malaka