In this article wirtten by Sumit Gupta, author of the book Building Web Applications with Python and Neo4j we will discuss and develop RESTful APIs for performing CRUD and search operations over our social network data, using Flask-RESTful extension and py2neo extension—Object-Graph Model (OGM). Let’s move forward to first quickly talk about the OGM and then develop full-fledged REST APIs over our social network data.
(For more resources related to this topic, see here.)
ORM for graph databases py2neo – OGM
We discussed about the py2neo in Chapter 4, Getting Python and Neo4j to Talk Py2neo. In this section, we will talk about one of the py2neo extensions that provides high-level APIs for dealing with the underlying graph database as objects and its relationships.
Object-Graph Mapping (http://py2neo.org/2.0/ext/ogm.html) is one of the popular extensions of py2neo and provides the mapping of Neo4j graphs in the form of objects and relationships. It provides similar functionality and features as Object Relational Model (ORM) available for relational databases py2neo.ext.ogm.Store(graph) is the base class which exposes all operations with respect to graph data models. Following are important methods of Store which we will be using in the upcoming section for mutating our social network data:
- Store.delete(subj): It deletes a node from the underlying graph along with its associated relationships. subj is the entity that needs to be deleted. It raises an exception in case the provided entity is not linked to the server.
- Store.load(cls, node): It loads the data from the database node into cls, which is the entity defined by the data model.
- Store.load_related(subj, rel_type, cls): It loads all the nodes related to subj of relationship as defined by rel_type into cls and then further returns the cls object.
- Store.load_indexed(index_name, key,value, cls): It queries the legacy index, loads all the nodes that are mapped by key-value, and returns the associated object.
- Store.relate(subj, rel_type, obj, properties=None): It defines the relationship between two nodes, where subj and cls are two nodes connected by rel_type. By default, all relationships point towards the right node.
- Store.save(subj, node=None): It save and creates a given entity/node—subj into the graph database. The second argument is of type Node, which if given will not create a new node and will change the already existing node.
- Store.save_indexed(index_name,key,value,subj): It saves the given entity into the graph and also creates an entry into the given index for future reference.
Refer to http://py2neo.org/2.0/ext/ogm.html#py2neo.ext.ogm.Store for the complete list of methods exposed by Store class.
Let’s move on to the next section where we will use the OGM for mutating our social network data model.
OGM supports Neo4j version 1.9, so all features of Neo4j 2.0 and above are not supported such as labels.
Social network application with Flask-RESTful and OGM
In this section, we will develop a full-fledged application for mutating our social network data and will also talk about the basics of Flask-RESTful and OGM.
Creating object model
Perform the following steps to create the object model and CRUD/search functions for our social network data:
- Our social network data contains two kind of entities—Person and Movies. So as a first step let’s create a package model and within the model package let’s define a module SocialDataModel.py with two classes—Person and Movie:
class Person(object): def __init__(self, name=None,surname=None,age=None,country=None): self.name=name self.surname=surname self.age=age self.country=country class Movie(object): def __init__(self, movieName=None): self.movieName=movieName
- Next, let’s define another package operations and two python modules ExecuteCRUDOperations.py and ExecuteSearchOperations.py.
- The ExecuteCRUDOperations module will contain the following three classes:
- DeleteNodesRelationships: It will contain one method each for deleting People nodes and Movie nodes and in the __init__ method, we will establish the connection to the graph database.
class DeleteNodesRelationships(object): ''' Define the Delete Operation on Nodes ''' def __init__(self,host,port,username,password): #Authenticate and Connect to the Neo4j Graph Database py2neo.authenticate(host+':'+port, username, password) graph = Graph('http://'+host+':'+port+'/db/data/') store = Store(graph) #Store the reference of Graph and Store. self.graph=graph self.store=store def deletePersonNode(self,node): #Load the node from the Neo4j Legacy Index cls = self.store.load_indexed('personIndex', 'name', node.name, Person) #Invoke delete method of store class self.store.delete(cls[0]) def deleteMovieNode(self,node): #Load the node from the Neo4j Legacy Index cls = self.store.load_indexed('movieIndex', 'name',node.movieName, Movie) #Invoke delete method of store class self.store.delete(cls[0])Deleting nodes will also delete the associated relationships, so there is no need to have functions for deleting relationships. Nodes without any relationship do not make much sense for many business use cases, especially in a social network, unless there is a specific need or an exceptional scenario.
- UpdateNodesRelationships: It will contain one method each for updating People nodes and Movie nodes and, in the __init__ method, we will establish the connection to the graph database.
class UpdateNodesRelationships(object): ''' Define the Update Operation on Nodes ''' def __init__(self,host,port,username,password): #Write code for connecting to server def updatePersonNode(self,oldNode,newNode): #Get the old node from the Index cls = self.store.load_indexed('personIndex', 'name', oldNode.name, Person) #Copy the new values to the Old Node cls[0].name=newNode.name cls[0].surname=newNode.surname cls[0].age=newNode.age cls[0].country=newNode.country #Delete the Old Node form Index self.store.delete(cls[0]) #Persist the updated values again in the Index self.store.save_unique('personIndex', 'name', newNode.name, cls[0]) def updateMovieNode(self,oldNode,newNode): #Get the old node from the Index cls = self.store.load_indexed('movieIndex', 'name', oldNode.movieName, Movie) #Copy the new values to the Old Node cls[0].movieName=newNode.movieName #Delete the Old Node form Index self.store.delete(cls[0]) #Persist the updated values again in the Index self.store.save_ unique('personIndex', 'name', newNode.name, cls[0]) - CreateNodesRelationships: This class will contain methods for creating People and Movies nodes and relationships and will then further persist them to the database. As with the other classes/ module, it will establish the connection to the graph database in the __init__ method:
class CreateNodesRelationships(object): ''' Define the Create Operation on Nodes ''' def __init__(self,host,port,username,password): #Write code for connecting to server ''' Create a person and store it in the Person Dictionary. Node is not saved unless save() method is invoked. Helpful in bulk creation ''' def createPerson(self,name,surName=None,age=None,country=None): person = Person(name,surName,age,country) return person ''' Create a movie and store it in the Movie Dictionary. Node is not saved unless save() method is invoked. Helpful in bulk creation ''' def createMovie(self,movieName): movie = Movie(movieName) return movie ''' Create a relationships between 2 nodes and invoke a local method of Store class. Relationship is not saved unless Node is saved or save() method is invoked. ''' def createFriendRelationship(self,startPerson,endPerson): self.store.relate(startPerson, 'FRIEND', endPerson) ''' Create a TEACHES relationships between 2 nodes and invoke a local method of Store class. Relationship is not saved unless Node is saved or save() method is invoked. ''' def createTeachesRelationship(self,startPerson,endPerson): self.store.relate(startPerson, 'TEACHES', endPerson) ''' Create a HAS_RATED relationships between 2 nodes and invoke a local method of Store class. Relationship is not saved unless Node is saved or save() method is invoked. ''' def createHasRatedRelationship(self,startPerson,movie,ratings): self.store.relate(startPerson, 'HAS_RATED', movie,{'ratings':ratings}) ''' Based on type of Entity Save it into the Server/ database ''' def save(self,entity,node): if(entity=='person'): self.store.save_unique('personIndex', 'name', node.name, node) else: self.store.save_unique('movieIndex','name',node.movieName,node)
- DeleteNodesRelationships: It will contain one method each for deleting People nodes and Movie nodes and in the __init__ method, we will establish the connection to the graph database.
Next we will define other Python module operations, ExecuteSearchOperations.py. This module will define two classes, each containing one method for searching Person and Movie node and of-course the __init__ method for establishing a connection with the server:
class SearchPerson(object):
'''
Class for Searching and retrieving the the People Node from server
'''
def __init__(self,host,port,username,password):
#Write code for connecting to server
def searchPerson(self,personName):
cls = self.store.load_indexed('personIndex', 'name', personName, Person)
return cls;
class SearchMovie(object):
'''
Class for Searching and retrieving the the Movie Node from server
'''
def __init__(self,host,port,username,password):
#Write code for connecting to server
def searchMovie(self,movieName):
cls = self.store.load_indexed('movieIndex', 'name', movieName, Movie)
return cls;
We are done with our data model and the utility classes that will perform the CRUD and search operation over our social network data using py2neo OGM.
Now let’s move on to the next section and develop some REST services over our data model.
Creating REST APIs over data models
In this section, we will create and expose REST services for mutating and searching our social network data using the data model created in the previous section.
In our social network data model, there will be operations on either the Person or Movie nodes, and there will be one more operation which will define the relationship between Person and Person or Person and Movie.
So let’s create another package service and define another module MutateSocialNetworkDataService.py. In this module, apart from regular imports from flask and flask_restful, we will also import classes from our custom packages created in the previous section and create objects of model classes for performing CRUD and search operations. Next we will define the different classes or services which will define the structure of our REST Services.
The PersonService class will define the GET, POST, PUT, and DELETE operations for searching, creating, updating, and deleting the Person nodes.
class PersonService(Resource):
'''
Defines operations with respect to Entity - Person
'''
#example - GET http://localhost:5000/person/Bradley
def get(self, name):
node = searchPerson.searchPerson(name)
#Convert into JSON and return it back
return jsonify(name=node[0].name,surName=node[0].surname,age=node[0].age,country=node[0].country)
#POST http://localhost:5000/person
#{"name": "Bradley","surname": "Green","age": "24","country": "US"}
def post(self):
jsonData = request.get_json(cache=False)
attr={}
for key in jsonData:
attr[key]=jsonData[key]
print(key,' = ',jsonData[key] )
person = createOperation.createPerson(attr['name'],attr['surname'],attr['age'],attr['country'])
createOperation.save('person',person)
return jsonify(result='success')
#POST http://localhost:5000/person/Bradley
#{"name": "Bradley1","surname": "Green","age": "24","country": "US"}
def put(self,name):
oldNode = searchPerson.searchPerson(name)
jsonData = request.get_json(cache=False)
attr={}
for key in jsonData:
attr[key] = jsonData[key]
print(key,' = ',jsonData[key] )
newNode = Person(attr['name'],attr['surname'],attr['age'],attr['country'])
updateOperation.updatePersonNode(oldNode[0],newNode)
return jsonify(result='success')
#DELETE http://localhost:5000/person/Bradley1
def delete(self,name):
node = searchPerson.searchPerson(name)
deleteOperation.deletePersonNode(node[0])
return jsonify(result='success')
The MovieService class will define the GET, POST, and DELETE operations for searching, creating, and deleting the Movie nodes. This service will not support the modification of Movie nodes because, once the Movie node is defined, it does not change in our data model. Movie service is similar to our Person service and leverages our data model for performing various operations.
The RelationshipService class only defines POST which will create the relationship between the person and other given entity and can either be another Person or Movie. Following is the structure of the POST method:
'''
Assuming that the given nodes are already created this operation
will associate Person Node either with another Person or Movie Node.
Request for Defining relationship between 2 persons: -
POST http://localhost:5000/relationship/person/Bradley
{"entity_type":"person","person.name":"Matthew","relationship": "FRIEND"}
Request for Defining relationship between Person and Movie
POST http://localhost:5000/relationship/person/Bradley
{"entity_type":"Movie","movie.movieName":"Avengers","relationship": "HAS_RATED"
"relationship.ratings":"4"}
'''
def post(self, entity,name):
jsonData = request.get_json(cache=False)
attr={}
for key in jsonData:
attr[key]=jsonData[key]
print(key,' = ',jsonData[key] )
if(entity == 'person'):
startNode = searchPerson.searchPerson(name)
if(attr['entity_type']=='movie'):
endNode = searchMovie.searchMovie(attr['movie.movieName'])
createOperation.createHasRatedRelationship(startNode[0], endNode[0], attr['relationship.ratings'])
createOperation.save('person', startNode[0])
elif (attr['entity_type']=='person' and attr['relationship']=='FRIEND'):
endNode = searchPerson.searchPerson(attr['person.name'])
createOperation.createFriendRelationship(startNode[0], endNode[0])
createOperation.save('person', startNode[0])
elif (attr['entity_type']=='person' and attr['relationship']=='TEACHES'):
endNode = searchPerson.searchPerson(attr['person.name'])
createOperation.createTeachesRelationship(startNode[0], endNode[0])
createOperation.save('person', startNode[0])
else:
raise HTTPException("Value is not Valid")
return jsonify(result='success')
At the end, we will define our __main__ method, which will bind our services with the specific URLs and bring up our application:
if __name__ == '__main__': api.add_resource(PersonService,'/person','/person/<string:name>') api.add_resource(MovieService,'/movie','/movie/<string:movieName>') api.add_resource(RelationshipService,'/relationship','/relationship/<string:entity>/<string:name>') webapp.run(debug=True)
And we are done!!! Execute our MutateSocialNetworkDataService.py as a regular Python module and your REST-based services are up and running. Users of this app can use any REST-based clients such as SOAP-UI and can execute the various REST services for performing CRUD and search operations.
Follow the comments provided in the code samples for the format of the request/response.
In this section, we created and exposed REST-based services using Flask, Flask-RESTful, and OGM and performed CRUD and search operations over our social network data model.
Using Neomodel in a Django app
In this section, we will talk about the integration of Django and Neomodel.
Django is a Python-based, powerful, robust, and scalable web-based application development framework. It is developed upon the Model-View-Controller (MVC) design pattern where developers can design and develop a scalable enterprise-grade application within no time.
We will not go into the details of Django as a web-based framework but will assume that the readers have a basic understanding of Django and some hands-on experience in developing web-based and database-driven applications.
Visit https://docs.djangoproject.com/en/1.7/ if you do not have any prior knowledge of Django.
Django provides various signals or triggers that are activated and used to invoke or execute some user-defined functions on a particular event.
The framework invokes various signals or triggers if there are any modifications requested to the underlying application data model such as pre_save(), post_save(), pre_delete, post_delete, and a few more.
All the functions starting with pre_ are executed before the requested modifications are applied to the data model, and functions starting with post_ are triggered after the modifications are applied to the data model. And that’s where we will hook our Neomodel framework, where we will capture these events and invoke our custom methods to make similar changes to our Neo4j database.
We can reuse our social data model and the functions defined in ExploreSocialDataModel.CreateDataModel.
We only need to register our event and things will be automatically handled by the Django framework. For example, you can register for the event in your Django model (models.py) by defining the following statement:
signals.pre_save.connect(preSave, sender=Male)
In the previous statement, preSave is the custom or user-defined method, declared in models.py. It will be invoked before any changes are committed to entity Male, which is controlled by the Django framework and is different from our Neomodel entity.
Next, in preSave you need to define the invocations to the Neomodel entities and save them.
Refer to the documentation at https://docs.djangoproject.com/en/1.7/topics/signals/ for more information on implementing signals in Django.
Signals in Neomodel
Neomodel also provides signals that are similar to Django signals and have the same behavior. Neomodel provides the following signals: pre_save, post_save, pre_delete, post_delete, and post_create.
Neomodel exposes the following two different approaches for implementing signals:
- Define the pre..() and post..() methods in your model itself and Neomodel will automatically invoke it. For example, in our social data model, we can define def pre_save(self) in our Model.Male class to receive all events before entities are persisted in the database or server.
- Another approach is using Django-style signals, where we can define the connect() method in our Neomodel Model.py and it will produce the same results as in Django-based models:
signals.pre_save.connect(preSave, sender=Male)
Refer to http://neomodel.readthedocs.org/en/latest/hooks.html for more information on signals in Neomodel.
In this section, we discussed about the integration of Django with Neomodel using Django signals. We also talked about the signals provided by Neomodel and their implementation approach.
Summary
Here we learned about creating web-based applications using Flask. We also used Flasks extensions such as Flask-RESTful for creating/exposing REST APIs for data manipulation. Finally, we created a full blown REST-based application over our social network data using Flask, Flask-RESTful, and py2neo OGM.
We also learned about Neomodel and its various features and APIs provided to work with Neo4j. We also discussed about the integration of Neomodel with the Django framework.
Resources for Article:
Further resources on this subject:
- Firebase [article]
- Developing Location-based Services with Neo4j [article]
- Learning BeagleBone Python Programming [article]
