Saturday, May 17, 2014

Initial clicks on the Usergrid Admin portal

The attempt of this post is to show you some of the areas in the Usergrid admin portal. Mainly to get you going on using Usergrid.
If you still do not have your local instance of "the BaaS framework you run" take a look at my previous post at: http://techviewofalearner.blogspot.com/2014/05/running-usergrid-get-started.html

The admin portal at first glance looks like this: 




As you will see, the portal has an Organization and Application set. In this case the Organization is 'test-organization' and the Application is 'test-app'.
I refuse to let you pass this point without an understanding that concept. So I am going to use a few images Sungju Jin (Usergrid commiter) used to explain the internals.

Usergrid supports multi-tenancy by design. The following diagram will give you an idea of what that really means:

Your organization will have many applications, you can think of the application as a sandbox. 
One organization can have many applications and those applications have the following structure:

And when accessing specific data, yeah that is right, go through the organization to your specific application:


Wait a minute, did I just mention 'collection' and 'entity' without explaining what that is ! hmph ! 
Relax, here it is below: 

A collection is basically like a table in sql, we can put items that we model into them (in a non relational DB mindset of course !). Things like 'Users', 'Books', 'Cars', 'Aliens', 'Ghosts' (like the one behind you, don't look back). 
Important thing to note here is that they were always in plural.
An entity is basically like a row in that table, a collection called 'Users' can have the following entity for example:
{

"uuid" : "<ID HERE>", 
"type" : "user",
"username" : "malaka",
"created": 138901448423
"modified" : 138901448423
"activated" : true

}

There you go, that is the basic structure. You have to make sure that is clear, after all the whole point of using a BaaS is not to focus on the back-end implementation. In Usergrid once you have your collections defined (we will see how in a bit) you can do the CRUD (create-read-update-delete) operations on them.
The table below explains how:


if your collection is items, you access it using  https://api.domain.io/my-org-id/my-app-id/items
Doing a GET request to that path gives you all the items available (returns an array of entities - that bulb that went of when you heard the word entities, that is proof you are settling into Usergrid). 
A POST will create a new entity, etc. as shown in the table above.

Now let's do this in Usergrid ! 

Click on 'App data' and then click on the 'Add collection' button:

Then you will see the following prompt: 



Put the name as 'item' (or anything you want) and press 'create'.  The new collection will show up in the list as you can see below:


As you can see it is now in ... sorry ? ah yeah plural. you are completing my sentences, you are really getting the hang of this. 

Now remember the http GET, POST, PUT etc. above ? So to create a new 'item' I have to do a ?? POST ! exactly. When doing this, we pass the attributes and values in  JSON key-value pairs. As the example below shows the value can be a string like "item 1" or even an int like 123. 


We have been given the option to validate our JSON, so why not use it. Click on 'Validate Json' and make sure your Json is valid. Then click 'run query'. Make sure that POST is selected above. 
This will add your new entity to the items collection. 

You can do a GET on the same to retrieve the item just added: 


To see additional details just click on the 'View Details' and you will see: 


The uuid (unique identifier), created time, modified time and other meta data is already added for you. 
You have now made your first collection and entity on Usergrid ! 


Regards, 
Malaka


More information on: 
Usergrid internals by Sungju Jin (Usergrid commiter) at : https://speakerdeck.com/sungjuly/apache-usergrid-internal

How to contribute to Usergrid by David Johnson (Usergrid commiter) at: http://www.slideshare.net/snoopdave/how-to-contribute-to-apache-usergrid










Tuesday, May 13, 2014

Running Usergrid - Get started




The Usergrid project is now open source and in the Apache incubation phase.

**[updated post available, the link can be found at the bottom after the project descriptions.]**

Setup environment for Usergrid: 
Requirements
JDK 1.6: (http://www.oracle.com/technetwork/java/javase/downloads/index.html)
Maven (http://maven.apache.org/): Download and install Maven using the instructions on the Apache Maven website

There are a few ways you can download the source from https://github.com/usergrid/usergrid/
1) Click on 'Download ZIP' or Clone the source and follow the instructions on the site to build it.

2) Create\Use your Github account to fork the repository into your own.

The second approach is recommended if you want to keep up to date with the latest code and/or want to contribute to the project. 


At a high level this it what the project structure looks like:

If we go into the SDK folder we can see the SDKs (software development kits) available for Usergrid. 


The stack contains the main project. The structure of the stack is as follows:

Here is a brief description of the projects from the readme files in the relevant folders.

Build-tools: Holds key configuration files for build plugins (Contains resources like the cassandra.yaml).

Config: Runtime config resources of Usergrid system are contained here.

Core: This contains the core services for Usergrid system.
Usergrid-core contains the persistence layer and shared utilities for powering the Usergrid service. The services layer is contained in Usergrid-services and exposes a higher-level API that's used by the Usergrid-rest web services tier.

Launcher: Usergrid Launcher is a graphical server launcher.
The launcher is a simple Java app that provides a GUI for running the server as a desktop app.

Mongo emulator: This provides the Mongo emulation layer for the Usergrid system  . This is still experimental and incomplete implementation of Mongo emulation layer so Mongo clients can connect to Usergrid. There are a number of drivers and tools for talking to Mongo DB, which has a data model that is very similar to that of Usergrid. In order to make it easier for people to get started quickly with Usergrid, it's desirable for them to be able to integrate it very quickly into their existing applications.
The goal here is to support the Mongo native wire protocol and to map enough of the query and CRUD operations so that this can happen.

Query-Validator: This test package verifies that the Usergrid REST API returns results and does paging and limits in a style similar to that of SqlLite.

Rest: This contains the REST web services for Usergrid system (Usergrid REST API Web App).
This installs as a web-app in Tomcat and has not been extensively tested in other web containers.

Services: This contains the Service layer for Usergrid system.
This is where most of the high-level functionality for accessing Usergrid data is performed. It functions as the glue between the REST API as exposed by Jersey and the entity management functionality in Usergrid-core. This is also where security and account management functionality is contained.
The security model of Usergrid is implemented with Apache Shiro: http://shiro.apache.org/
The services model for Usergrid involves converting the REST web service requests into collection queries and routing these to a set of collection handlers.    This provides a "virtual collection" layer on top of the actual collections that are exposed in the persistence tier.  This makes it possible to:
- implement logic behaviour on top of persistent objects
- enforce object and property visibility

Test-utils: This contains the test utilities for Usergrid system as implied by the name.

Tools: This contains the command line tools for Usergrid system as implied by the name.
The Usergrid Tools are a set of Java programs that can be run from a command-line console.

Websocket: This contains the WebSocket API for Usergrid system. It is still an experimental (incomplete) implementation of Websocket API.  Websockets info can be found at http://en.wikipedia.org/wiki/WebSockets . The Websocket API will allow for asynchronous communications with the Usergrid API. OAuth credentials are put in the query string of the initial GET request since the JavaScript Websocket API provides no mechanism for specifying header contents.
Once a connection is established, there are two mechanisms of usage.
Async Request/Response
The first is to simply access the API the same way as the REST interface provides, except over the open websocket connection. The benefit is there is much less overhead than with the REST API since there is no processing of individual HTTP requests and authentication credentials are checked at the time the websocket is opened as opposed to per-request as is necessary with the stateless REST API.

Subscriptions
The second is to issue subscribe requests against entities and collections and be notified asynchronously as those are updated. Subscriptions against the activity inbox and message queue collections for either users, groups, or the entire application allow connected applications to get real-time updates.
Connecting to an inbox or queue in the websocket URL will automatically start a subscription to that collection. For example, the following websocket URL will listen to a specific user's inbox:
ws://api.usergrid.com:8088/chatapp/users/johndoe/inbox
This is the real-time equivalent of making a regular REST GET request to:
http://api.usergrid.com/chatapp/users/edanuff/inbox
Usergrid listens to websockets on an alternate port than it does standard HTTP requests. Although websockets can coexist on the same ports 80 or 443, Tomcat and other Java servlet containers don't do a particularly great job of handling open async socket connections despite what they claim. For our purposes, we're using a bare metal Netty websocket server that listens on port 8088. This is going to be much more performant that trying to integrate it into Tomcat and allows us to move the websocket servers onto different machines.

Getting started: 

Once you have the source downloaded (downloaded/cloned/forked) you can to the the stack folder and run the Maven command: 

(you need to have your java path, maven path and maven repository setup for the command to be successful)


This will download the required dependencies to the Maven repository which are needed to build the Usergrid project.
Once that is done, we will go into the launcher project (see the description of launcher project above) and run that jar file with the command:


[Since the Usergrid source has been updated please switch to http://malakablogs.blogspot.com/2015/06/running-usergrid-improved-version-with.html and continue reading.]

When this jar file runs, what happens is that an instance of the Grizzly server is made inside the target folder. This is so that users can host Usergrid instantly without the need to set up servers. 
It gets better, users do not even have to setup the Cassendra back-end as an instance of that will be made as well for you as you will see below:
(If however after you have got the hand-on of Usergrid you want to setup your own servers and Cassendra instances refer the documents at: http://usergrid.incubator.apache.org/ )

When the jar file is run you will see the following Graphical User Interface (GUI):


As you can see 'Start Database With Server' is already checked - this creates an instance of the Grizzly server onto which Usergrid will be deployed. 
'Initialize Database on Start' will create an instance of the Cassendra back-end for you to play around as well. (Keep in mind however that unlike a proper instance, this stops when you stop the GUI above).

The 'Console URL' is by default pointer to  http://apigee.github.io/usergrid-portal/. You can go ahead and use that. However later on you can host the 'Portal' web project (see project structure) in your own server and point to that (e.g.: http://localhost/usergrid-portal/). 


   Go ahead and press the Play button. The light will change color from red to orange to green. Green means that the setup was successful and you can now use the system. 



You now have the Usergrid console to play around and learn !  Don't forget to have fun doing it ! 

Regards, 
Malaka 

Saturday, April 12, 2014

Usergrid sub-projects information

This is a small article that uses the readMe files in the stack project of Apache Usergrid so that enthusiasts can get the information from one location without going through the file structure. 

Keep in mind that the files may have been changed by the time you are reading this though.
This will help you get the basic idea. Enjoy !

User- Grid- Stack

The user-grid-stack consists of a parent maven project with the following modules.

Modules:

Built-tools

Holds key configuration files for build plugins (Contains resources like the checkstyle.xml).

 

Config

Runtime config resources of Usergrid system are contained here.

 

Core

This contains the core services for Usergrid system.
Usergrid-core contains the persistence layer [m1] [Mm2] and shared utilities for powering the Usergrid service. The services layer is contained in Usergrid-services and exposes a higher-level API that's used by the Usergrid-rest web services tier.

Launcher

Usergrid Launcher is a graphical server launcher.
The launcher is a simple Java app that provides a GUI for running the server as a desktop app.

 

Mongo emulator

This provides the Mongo emulation layer for the Usergrid system[m3] [Mm4] . This is still experimental and incomplete implementation of Mongo emulation layer so Mongo clients can connect to Usergrid.
There are a number of drivers and tools for talking to Mongo DB, which has a data model that is very similar to that of Usergrid. In order to make it easier for people to get started quickly with Usergrid, it's desirable for them to be able to integrate it very quickly into their existing applications.
The goal here is to support the Mongo native wire protocol and to map enough of the query and CRUD operations so that this can happen.

Rest

This contains the REST web services for Usergrid system (Usergrid REST API Web App).
This installs as a web-app in Tomcat and has not been extensively tested in other web containers.
See Usergrid-standalone for an example of running inside Grizzly.

Services

This contains the Service layer for Usergrid system.
This is where most of the high-level functionality for accessing Usergrid data is performed. It functions as the glue between the REST API as exposed by Jersey and the entity management functionality in Usergrid-core. This is also where security and account management functionality is contained.
The security model of Usergrid is implemented with Apache Shiro: http://shiro.apache.org/
The services model for Usergrid involves converting the REST web service requests into collection queries and routing these to a set of collection handlers.  [m5] [Mm6] This provides a "virtual collection" layer on top of the actual collections that are exposed in the persistence tier.  This makes it possible to:
- implement logic behaviour on top of persistent objects
- enforce object and property visibility
- perform fine-grained permission checks

 

Test-utils

This contains the test utilities for Usergrid system as implied by the name.

 

Tools

This contains the command line tools for Usergrid system as implied by the name.
The Usergrid Tools are a set of Java programs that can be run from a command-line console.

Websocket

This contains the WebSocket API for Usergrid system. It is still an experimental (incomplete) implementation of Websocket API.  Websockets info can be found at http://en.wikipedia.org/wiki/WebSockets
The Websocket API will allow for asynchronous communications with the Usergrid API.
OAuth credentials are put in the query string of the initial GET request since the JavaScript Websocket API provides no mechanism for specifying header contents.
Once a connection is established, there are two mechanisms of usage.
Async Request/Response
The first is to simply access the API the same way as the REST interface provides, except over the open websocket connection. The benefit is there is much less overhead than with the REST API since there is no processing of individual HTTP requests and authentication credentials are checked at the time the websocket is opened as opposed to per-request as is necessary with the stateless REST API.

Subscriptions
The second is to issue subscribe requests against entities and collections and be notified asynchronously as those are updated. Subscriptions against the activity inbox and message queue collections for either users, groups, or the entire application allow connected applications to get real-time updates.
Connecting to an inbox or queue in the websocket URL will automatically start a subscription to that collection. For example, the following websocket URL will listen to a specific user's inbox:
ws://api.usergrid.com:8088/chatapp/users/johndoe/inbox
This is the real-time equivalent of making a regular REST GET request to:
http://api.usergrid.com/chatapp/users/edanuff/inbox
Usergrid listens to websockets on an alternate port than it does standard HTTP requests. Although websockets can coexist on the same ports 80 or 443, Tomcat and other Java servlet containers don't do a particularly great job of handling open async socket connections despite what they claim. For our purposes, we're using a bare metal Netty websocket server that listens on port 8088. This is going to be much more performant that trying to integrate it into Tomcat and allows us to move the websocket servers onto different machines.


Sunday, January 12, 2014

NSInvalidArgumentException that propagated all the way to my main class.

If you are here, you  probably got a NSInvalidArgumentException. Below is what I got, and even if your one is not the same, hopefully the explanation will help you solve yours:



Here is why this came all the way up:
- My application uses a documentInteractionController

  • It resides in a class
  • Helps to figure out 3rd party applications on my iOS device I can use to open certain files from my application
  • once it does and I go back to my application... bang ! my app exists, mugshot of the culprit is above
This is because the documentInteractionController fires callbacks, and unfortunately in my application the class was released and no longer available. Since the class is no where to be found, the OS cannot call the callback methods and we get the above error. (callbacks are methods that fire after certain events, in this case the method "didEndSendingToApplication" fires to let us know that the 3rd party application got my file)

-To rectify:
  • I had to retain the documentInteractionController (and also the parent view due to some specifics of my application)
  • Release it after the callbacks were fired.
Read a bit on memory management in iOS and ARC (automatic reference counting)


You basically the 'retain' keyword to keep your objects around and then 'release' it after you are done. 
Make sure you cover all paths if you do so, you are interfering with the memory management, so cover your tracks well and avoid memory leaks. Even though you might not see any visible side effects,  we should always write quality code :) 

Hope this helps someone :)

iOS 7 has added a background colour to your table cell


In iOS 7 we noticed a white background for the UITableView we were using. Whereas on iOS 6 the same code gave us a transparent background, thus enabling the background image to be visible as required by the App's owner.
This was a change done in iOS 7, so if you want to support it, do a version check and something similar to the following:  

//  (iOS 6 cell is transparent by default not on iOS 7).

      //do a version check, if it is iOS 7 run the below code
        cell.backgroundColor = [UIColor clearColor];
    


You can use a callback method such as the one below to add your code if you are not sure where to place it:

- (UITableViewCell *) tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath 
{
     if([AppDelegate iSIOS7OrAbove]){  // iSIOS7OrAbove is not a method that comes out of the box. write the logic you want in a custom method to do the check.
        cell.backgroundColor = [UIColor clearColor];
    }

}


Hope this helped someone :)