- [Dev Cloud Essentials] How to never worry about your WordPress hosting?
- [Dev Cloud Essentials] Create affordable brand monitoring process in minutes
What is brand monitoring?
Let’s say you have just opened an online store. As every business owner you care about opinion that people have about your store. Anytime someone has a question, doubt, complain and expresses it on your facebook fanpage or directly on your website – you politely answer. You simply care, because you want your customers to be satisfied.
What if some of the questions, doubts or complains are expressed in other parts of the internet, without appropriate tags? If you’d know about it, you’d still do the same – politely answer. But you are not aware. You’re busy with your online store – why would you spend a whole day in the internets looking for bad opinions?
Brand monitoring is basically that – it looks for so called “mentions” and notify you. You save time on looking by yourself and expand your territory to other areas of the internet than your website. It helps to adapt to the real live feedback and in the end deliver better value to receivers.
Do you wonder why the hell am I telling you this?
Am I looking for a problem for my favorite tools?
I have a days like this sometimes. I just sit in the corner and I think. Technology nowadays is helping us in so many ways – how could I use the parts I know to help me? To make something cheaper? Or make something less time consuming?
Brand monitoring is something that every business owner, content creator, public performer and anyone doing something for other people does.
Plenty of them manually, they check social media, google their own brand, observe comments on their website.
Some of them using SaaS (Software as a Service) tools. There are some really great and powerful tools out there that will automate brand monitoring and save us plenty of time.
Those tools are not free obviously. Most of the plans start with at least $50. It is a reasonable price, if time spent for manual brand monitoring and information aggregation is more valuable than $50.
So I was sitting and thinking – Is there a way I could use cloud solutions to make brand monitoring even cheaper? Would it even make sense?
I have found the way
Dude, why would you even try to solve the problem that already has been solved? Well, why would anyone try to use a new tool for existing problems?
Maybe to see if the new tool can fix the old problems better or cut the costs.
Maybe just for fun.
You don’t have to pick only one of the above.
I decided to create a simple brand monitoring automation using only cloud solutions. I did that. Yep. And I managed to go down to monthly costs somewhere around $20.
Oh, and a implementation of the solution took me about 30 minutes.
The functionality provided in these 30 minutes is nothing comparing to the professional tools offered for bigger money. But it’s good enough for a smaller amount of money. Let’s compare.
|Brand24||Custom Cloud solution|
|5 keywords||1 in current implementation – not hard to extend|
|50K mentions / mo||Unlimited|
|5 users||No authentication in current implementation|
|Update every 12h||Update every 1h|
|1 month archive||As big as you like|
|All the sources possible||Just twitter now|
|Slack Integration||No – but can be easily added|
Doesn’t look so bad. Let’s see what I actually did. I used a few cool cloud features – have a look!
Prerequisites: Azure subscription (visit my first post in Dev Cloud Essentials for instructions how to get a subscription), twitter account.
The whole process is divided into three areas that I will guide you through separetly:
- I need to gather tweets containing my keyword
- Calculate sentiment of each of them
- Display mentions somewhere – let’s say it will be a web application
Blocks I need to implement
- something to monitor twitter feed and get the tweets with my brand in it
- a store all the raw and processed tweets
- something that will tell me if the tweet was positive, negative or neutral
- some kind of service that will take the saved tweets and run then through the service calculating the tweet sentiment
- an endpoint that will provide ready-to-display mentions in the web app
- a container for my web application
Let’s have a look how we could achieve those in Azure.
How to get those tweets?
Monitoring twitter feed is a first thing to take care of. Normally we would use some kind of a service running in the background. Yet, Azure provides us with another solution. There is a service called Logic Apps that will give me something like that out of the box for a little money.
Furthermore, to store the tweets I obviously need some kind of a database – in this case a very simple document db would be sufficient. Well, fortunately there is a PaaS (Platform as a Service) solution called Cosmos DB that fits my requirements.
Do I need my own neural network?
Calculating sentiment of the tweets seems like a big job. Fortunately, we don’t have to create our own neural network, because there are ready services providing text analysis. On Azure it’s called Cognitive Services. Let’s use it.
Alright. How should I now connect those stored tweets with text analysis, I need some piece of code that would glue it together and create ready elements that I can display ob the website.
That’s sounds like a perfect use case for Functions – which are just a piece of code that can react on certain triggers, process the payload and deliver output however we want. I will write the code that will be triggered on a new document in tweets database, send it to text analysis and save the results.
Let’s not create another web API
Normally to extract the contents of the database to the outside world in a acceptable way I would need to create some kind of web API. I have a better idea though – kind of a hack.
There is a native Azure Search service that can be continually indexed with new data put into the designated database. It provides one simple endpoint that outputs the data from the database. Sounds like a good solution to not write the entire web API!
At the end I need to write a simple website and put it somewhere in the internet. Another Azure PaaS service called Web Apps is something exactly what we need here. I can create a container for a website and easily deploy my code there, not worrying about the whole confrontational nonsense.
Perfect! We got all written out. Let’s do it.
In this part we need to take care of twitter monitoring and storing the tweets containing the keyword that interests us.
- Go to Azure portal and click on “Create a resource” and choose CosmosDB.
- When it’s done, enter an unique name, select API you will use to communicate with the DB, fill the rest of the details however you prefer.
- After deploying the resource – go to the created resource and go to overview and click on data explorer.
- In data explorer click on “New collection” and name your collection and database (in my case it was “Monitoring” for the DB and “Tweets” for the collection.
- Click again on “Create a resource” and choose Logic App this time.
- After creating logic app go to the resource overview and click on “Edit”.
- Now, choose a trigger: “When a new tweet is posted”.
- After filling connection details and search text, click “New step”.
- When you’re done, choose “Create or update document” from CosmosDB actions and choose the created DB.
- Next, fill the step details like on my screenshot.
- Save Logic App. Alright, now every tweet with my brand will be saved to the CosmosDB collection. Cool!
This is quite a fun part. What do you think, how difficult is to use AI to calculate the sentiment of the tweet? Have a look.
- Click “Create a new resource” again in the left top corner of the screen.
- Choose the Text Analysis (Cognitive Services) and create it.
- Now after creating the resource go to it and take the key from the Keys section. You will use it in a second.
- Click on “create resource” button again. Choose Function App. After filling the details for the new resource and creating to to the newly created function app and click little plus icon next to the “Functions” section.
- Choose CosmosDB trigger.
- Fill the details with your CosmosDB collection information.
- After saving the function, go to the integrate section and click on a big plus under Outputs column on the top right.
- Set it to the same collection as in the trigger- in my case that’s “Tweets”. Have a look at my settings.
- Save and go to the Function App Settings.
- Insert your cognitive services key into the app settings – you’ll use it later in the function.
- Now the fun part, we have to write the logic responsible for calculating the sentiment of the tweet.
- The code is not complicated – you need to iterate through the documents that came from CosmosDB, send a few requests to cognitive services with a tweets text then save it to output variable. Don’t want to write that code? Feel free to use the one in my github repo.
- If you use my code – after inserting all the three files, you have to go to: https://[YOUR_FUNCTION_NAME].scm.azurewebsites.net/DebugConsole and run npm install in the catalog containing package.json
Probably a bit hacky, but it works! Would you prefer to write another boring API?
- Go to the CosmosDB and find the section Add Azure Search. Click on it and and click on create new search service.
- When you’re done – choose the newly created search service. Now go to the next section of creation wizard.
- Choose your CosmosDB and your collection of tweets. Important note: make sure you have an existing document in the collection already (add some tweet and wait for it to get stored). Otherwise you will get an error, that the service cannot extract fields for the search index.
- After it’s done, choose the fields for the search index to contain.
- Finally, the last step of the wizard is setting up the indexer (service that will insert data to the search index). Set it to run hourly.
- After you create it, remember to set the CORS options to every origin (or the origin of your app that will use it). Otherwise you will have problems with getting the mentions.
- Right on! Now, create the service. Wait a little and it’s done. Congratulations. you have the endpoint you can use to obtain the brand mentions with the sentiment calculation.
- If you want to display it in a nice way (like I showed you in the beginning) – you can create very simple website. Click on “Create a new resource again” and choose web app. I decided to create a really simple and dumb react app that will just display the list of mentions from Azure Search endpoint. I used the boilerplate from my dear friend – visit his repo. If you don’t want to mess with website development now, just use my code from github!
- The code of the website can be deployed in many ways – the fastest would be using Local Git repository (simply commit and push the code to the git endpoint created in Azure Portal) or just use the FTP (you have to setup deployments credentials in both cases).
- That’s it! By now you should have a working process of brand monitoring and a nice, working website displaying the mentions. Enjoy!
The cloud can be used to not only to solve the the big, enterprise problems – unique for a specific business. We can use it to just make our daily routine easier or cheaper.
I started using that tool for brand monitoring of one of my projects on Twitter. It helps me to learn how to interact with Twitter people. Have a look at the result.
Maybe in time I will eventually migrate to some SaaS service. For larger scale it’s more cost effective and gives a lot more functionality. For now it’s sufficient enough to:
- help me with a proper communication on one of the social media
- monitor the reach of my brand presence
- calculate my needs for a more expensive tool
Note: I could go down with the monthly price for the tool I created with replacing CosmosDB with some other database, but that would cost me a bit more time in creating the solution. Time worth more than a few euros per month. Feel free to experiment on your own!
How can I put it into practice?
If you want to start brand monitoring on Twitter for a small money – you can grab all of the code I created in this proof of concept and use it or modify it.
With my instructions and my code you should be able to create a working solution in no time!
Today I also showed you possibilities of a few cloud services. I showed you how it can bring benefit even in quite small areas of life. Think about the activities you spend time on. Maybe some of them can be easier, cheaper or less pain in the back? My example is just one way of using cloud in making business a bit less time/money consuming.
If you don’t need brand monitoring – you can try to adjust this solution to any other case. Maybe you’ll find some other problem fitting a similar process. If so – write a comment and let’s try to solve it together.
Most of all, I hope you see that it’s fun to experiment with the cloud capabilities and that we can create a solutions that could solve our existing problems in a cheaper and easier way.
See you around!