In recent years more and more providers have been offering managed cloud-based database services. They have infrastructure needs, Software patching, backup and many other administrative tasks. AWS also offers a range of database solutions and one of the most popular is DynamoDB. In this blog, we will be discussing Dynamo DB, the new and fast way of retrieving data using the AWS platform.
What is DynamoDB?
DynamoDB is a fully managed NoSQL database service offered by AWS. It works on key-value pairs and other data structure documents provided by Amazon. No SQL databases are non-tabular databases and store databases differently than relational tables. Dynamo DB only requires a primary key and does not require a schema to create a table. Hence it can store any amount of data serve any amount of traffic. You can even expect good performance even when it scales up, It’s pretty simple to learn and a small API that follows the key-value method to store, access and perform advanced data retrieval,
DynamoDB comprises 3 Fundamental Unit
- Attribute: It is the simplest element in Dynamo DB that store data without any further division. It is a single filed that is attached to an item. E.g. Employee Name
- Item: An Item hold the set of Attribute. It uniquely identifies your entry in your table. For example, An Item in Employees records will identify a single employee
- Table: An Table holds the set of Items. For example, Employee record you will have the employee name, employee phone no, employee id. All such items will be stored in the table.
Why DynamoDB?
The main problem that NoSQL database aims to solve typically revolves around issues of scale when the database is no longer fixed on the single MySQL Server or single machine cannot handle query load. Some sort of sharding and replication is required. The NoSQL database was designed from the ground up to be distributed to handle large data volume. It allows us to model data close to what the application required.
The RDBMS usually force the domain model into the structure of Table and columns whereas NoSQL Database offers you to store data as documents or as key-value pair or even as cross structure. NoSQL is semantic richness.
DynamoDB VS Other DB Services
| Features | DynamoDB | RDS | RedShift | Aurora |
|---|---|---|---|---|
| Scaling | Scales Seamlessly | Relatively Easy to Scale | More Complexity on Scaling | Relatively Easy to Scale |
| Storage | Unlimited Storage | 64TB | 2PB | 64TB with RDS |
| Pricing | Pay Per Use Model | Generally cheaper than others | On Demand model with added cost | Pricing depends on RDS |
| Maintainance | Maintained by AWS | Maintained by AWS | Requires more maintainance | No Maintainance required |
Sample Application in DynamoDB
1. Click on Create Table in Dynamo DB Service
2. Enter Table Name and Primary Key and then Click Create
Once you click to create a table is created. You will have Options such as Overview, Items, Metrics, Alarm, Indexes, Capacity
You can click on Item and the add Item. In this, I have added 3 Items and in order to search, you can simply add a filter and then you can click on Start Search.
Query In DynamoDB
Create Table
var AWS = require("aws-sdk");
AWS.config.update({region: "us-east-2"});
var dynamodb = new AWS.DynamoDB({apiVersion: "2012-08-10"});
var documentClient = new AWS.DynamoDB.DocumentClient();
var params = {
AttributeDefinitions: [
{
AttributeName: "employeeID",
AttributeType: "S"
},
{
AttributeName: "email",
AttributeType: "S"
}
],
KeySchema: [
{
AttributeName: "employeeID",
KeyType: "HASH"
},
{
AttributeName: "email",
KeyType: "RANGE"
}
],
ProvisionedThroughput: {
ReadCapacityUnits: 1,
WriteCapacityUnits: 1
},
TableName: "tb_employee"
};
dynamodb.createTable(params, function(err, data) {
if (err) {
console.log("Error", err);
} else {
console.log("Table Created", data);
}
});
Insert Into Table
var AWS = require("aws-sdk");
AWS.config.update({region: "us-east-2"});
var dynamodb = new AWS.DynamoDB({apiVersion: "2012-08-10"});
var documentClient = new AWS.DynamoDB.DocumentClient();
var params = {
TableName: "tb_employee",
Item: {
"employeeID": "2",
"email": "[email protected]",
}
};
documentClient.put(params, function(err, data) {
if (err) {
console.error("Can't add Employee");
} else {
console.log("Succeeded adding an employee ");
}
});
GET ITEM
USING KEY
var AWS = require("aws-sdk");
AWS.config.update({region: "us-east-2"});
var dynamodb = new AWS.DynamoDB({apiVersion: "2012-08-10"});
var documentClient = new AWS.DynamoDB.DocumentClient();
var params = {
Key: {
"employeeID":{"S": "1"},
"email": {"S": "[email protected]"}
},
TableName: "tb_employee"
};
var result = dynamodb.getItem(params).promise().then(el=>{
console.log("Result:",el)
}).catch(err=>{
console.log("Key Does not exist");
})
USING QUERY
var AWS = require("aws-sdk");
AWS.config.update({region: "us-east-2"});
var dynamodb = new AWS.DynamoDB({apiVersion: "2012-08-10"});
var documentClient = new AWS.DynamoDB.DocumentClient();
var params = {
KeyConditionExpression: 'employeeID = :employeeID',
ExpressionAttributeValues: {
':employeeID': {'S': '1'}
},
TableName: "tb_employee"
};
var result = dynamodb.query(params).promise().then(el=>{
console.log("Result:",JSON.stringify(el))
}).catch(err=>{
console.log("Key Does not exist");
})
UPDATE ITEM
var AWS = require("aws-sdk");
AWS.config.update({region: "us-east-2"});
var dynamodb = new AWS.DynamoDB({apiVersion: "2012-08-10"});
var documentClient = new AWS.DynamoDB.DocumentClient();
var params = {
Key:{"employeeID": "1", "email": "[email protected]"},
UpdateExpression: 'set age = :updatedage',
ExpressionAttributeValues: {
':updatedage': '25'
},
TableName: "tb_employee"
};
var result = documentClient.update(params).promise().then(el=>{
console.log("Result:",JSON.stringify(el))
}).catch(err=>{
console.log(err)
console.log("Key Does not exist");
})
CSV TO DynamoDB
You can even import bulk Data From CSV to Dynamo DB. Just replace schema and CSV in the below code and it will work perfectly fine. The maximum number of chunks allowed is 25 in the below code.
CSV FILE
"id","email","age" "5","Rajesh","43" "6","Sameer","54" "7","Tanker","23"
NodeJS CODE
var AWS = require("aws-sdk");
AWS.config.update({region: "us-east-2"});
var dynamodb = new AWS.DynamoDB({apiVersion: "2012-08-10"});
var fs = require('fs');
var parse = require('csv-parse');
var async = require('async');
var csv_filename = "<Path to CSV File>";
rs = fs.createReadStream(csv_filename);
parser = parse({
columns : true,
delimiter : ','
}, function(err, data) {
var split_arrays = [], size = 25;
while (data.length > 0) {
split_arrays.push(data.splice(0, size));
}
data_imported = false;
chunk_no = 1;
async.each(split_arrays, function (data, callback) {
let params = {
RequestItems: {
tb_employee: []
}
};
for (let el of data) {
let obj = {
PutRequest: {
Item: {
"employeeID": { "S": el.id },
"email": { "S": el.email },
"age": { "S": el.age }
}
}
};
params.RequestItems.tb_employee.push(obj);
}
dynamodb.batchWriteItem(params, function (err, res, cap) {
console.log('done going next');
if (err == null) {
console.log('Success chunk #' + chunk_no);
data_imported = true;
} else {
console.log(err);
console.log('Fail chunk #' + chunk_no);
data_imported = false;
}
chunk_no++;
callback();
});
}, function () {
console.log('all data imported....');
});
});
rs.pipe(parser);
If you are interested in more such blogs then Please subscribe to our Blog infohubblog.com.
Reference :
Leave a comment