[Dev Cloud Essentials] Create affordable brand monitoring process in minutes

This entry is part 2 of 2 in the series Dev Cloud Essentials
Reading Time: 8 minutes

 

What is brand monitoring?

Let’s say you have just opened an online store. As every business owner you care about opinion that people have about your store. Anytime someone has a question, doubt, complain and expresses it on your facebook fanpage or directly on your website – you politely answer. You simply care, because you want your customers to be satisfied.

Source: https://tenor.com/search/find-you-gifs

What if some of the questions, doubts or complains are expressed in other parts of the internet, without appropriate tags? If you’d know about it, you’d still do the same – politely answer. But you are not aware. You’re busy with your online store – why would you spend a whole day in the internets looking for bad opinions?

Brand monitoring is basically that – it looks for so called “mentions” and notify you. You save time on looking by yourself and expand your territory to other areas of the internet than your website. It helps to adapt to the real live feedback and in the end deliver better value to receivers.

Do you wonder why the hell am I telling you this?

Am I looking for a problem for my favorite tools?

I have a days like this sometimes. I just sit in the corner and I think. Technology nowadays is helping us in so many ways – how could I use the parts I know to help me? To make something cheaper? Or make something less time consuming?

Source: https://giphy.com/gifs/thinking-SabSYEpsVh0di

Brand monitoring is something that every business owner, content creator, public performer and anyone doing something for other people does.

Plenty of them manually, they check social media, google their own brand, observe comments on their website.

Some of them using SaaS (Software as a Service) tools. There are some really great and powerful tools out there that will automate brand monitoring and save us plenty of time.

Those tools are not free obviously. Most of the plans start with at least $50. It is a reasonable price, if time spent for manual brand monitoring and information aggregation is more valuable than $50.

So I was sitting and thinking – Is there a way I could use cloud solutions to make brand monitoring even cheaper? Would it even make sense?

I have found the way

Dude, why would you even try to solve the problem that already has been solved? Well, why would anyone try to use a new tool for existing problems?

Maybe to see if the new tool can fix the old problems better or cut the costs.

Maybe just for fun.

You don’t have to pick only one of the above.

Source: http://popkey.co/m/0DaVb-wow-eureka-amazed-shocked-shocking

I decided to create a simple brand monitoring automation using only cloud solutions. I did that. Yep. And I managed to go down to monthly costs somewhere around $20.

Oh, and a implementation of the solution took me about 30 minutes.

The functionality provided in these 30 minutes is nothing comparing to the professional tools offered for bigger money. But it’s good enough for a smaller amount of money. Let’s compare.

Brand24 Custom Cloud solution
5 keywords 1 in current implementation – not hard to extend
50K mentions / mo Unlimited
5 users No authentication in current implementation
Update every 12h Update every 1h
1 month archive As big as you like
Data analysis Nope
Sentiment analysis Yep
Mobile app Nope
All the sources possible Just twitter now
Slack Integration No – but can be easily added

Doesn’t look so bad. Let’s see what I actually did. I used a few cool cloud features – have a look!

Cloud up!

Prerequisites: Azure subscription (visit my first post in Dev Cloud Essentials for instructions how to get a subscription), twitter account.

The whole process is divided into three areas that I will guide you through separetly:

  • I need to gather tweets containing my keyword
  • Calculate sentiment of each of them
  • Display mentions somewhere – let’s say it will be a web application

Blocks I need to implement

  • something to monitor twitter feed and get the tweets with my brand in it
  • a store all the raw and processed tweets
  • something that will tell me if the tweet was positive, negative or neutral
  • some kind of service that will take the saved tweets and run then through the service calculating the tweet sentiment
  • an endpoint that will provide ready-to-display mentions in the web app
  • a container for my web application

Let’s have a look how we could achieve those in Azure.

How to get those tweets?

Monitoring twitter feed is a first thing to take care of. Normally we would use some kind of a service running in the background. Yet, Azure provides us with another solution. There is a service called Logic Apps that will give me something like that out of the box for a little money.

Furthermore, to store the tweets I obviously need some kind of a database – in this case a very simple document db would be sufficient. Well, fortunately there is a PaaS (Platform as a Service) solution called Cosmos DB that fits my requirements.

Do I need my own neural network?

Calculating sentiment of the tweets seems like a big job. Fortunately, we don’t have to create our own neural network, because there are ready services providing text analysis. On Azure it’s called Cognitive Services. Let’s use it.

Alright. How should I now connect those stored tweets with text analysis, I need some piece of code that would glue it together and create ready elements that I can display ob the website.

That’s sounds like a perfect use case for Functions – which are just a piece of code that can react on certain triggers, process the payload and deliver output however we want. I will write the code that will be triggered on a new document in tweets database, send it to text analysis and save the results.

Let’s not create another web API

Normally to extract the contents of the database to the outside world in a acceptable way I would need to create some kind of web API. I have a better idea though – kind of a hack.

There is a native Azure Search service that can be continually indexed with new data put into the designated database. It provides one simple endpoint that outputs the data from the database. Sounds like a good solution to not write the entire web API!

At the end I need to write a simple website and put it somewhere in the internet. Another Azure PaaS service called Web Apps is something exactly what we need here. I can create a container for a website and easily deploy my code there, not worrying about the whole confrontational nonsense.

Perfect! We got all written out. Let’s do it.

Gather tweets

In this part we need to take care of twitter monitoring and storing the tweets containing the keyword that interests us.

  1. Go to Azure portal and click on “Create a resource” and choose CosmosDB.Cosmos Db
  2. When it’s done, enter an unique name, select API you will use to communicate with the DB, fill the rest of the details however you prefer. Cosmos Db
  3. After deploying the resource – go to the created resource and go to overview and click on data explorer. Cosmos Db
  4. In data explorer click on “New collection” and name your collection and database (in my case it was “Monitoring” for the DB and “Tweets” for the collection. Cosmos Db collection
  5. Click again on “Create a resource” and choose Logic App this time.
  6. After creating logic app go to the resource overview and click on “Edit”. Logic App
  7. Now, choose a trigger: “When a new tweet is posted”. Logic App triggers
  8. After filling connection details and search text, click “New step”. Logic App twitter
  9. When you’re done, choose “Create or update document” from CosmosDB actions and choose the created DB. Logic App cosmos db
  10. Next, fill the step details like on my screenshot. Logic app dynamic content
  11. Save Logic App. Alright, now every tweet with my brand will be saved to the CosmosDB collection. Cool!

Process tweets

This is quite a fun part. What do you think, how difficult is to use AI to calculate the sentiment of the tweet? Have a look.

  1. Click “Create a new resource” again in the left top corner of the screen.
  2. Choose the Text Analysis (Cognitive Services) and create it.
  3. Now after creating the resource go to it and take the key from the Keys section. You will use it in a second.
  4. Click on “create resource” button again. Choose Function App. After filling the details for the new resource and creating to to the newly created function app and click little plus icon next to the “Functions” section. Function app
  5. Choose CosmosDB trigger. Function app available triggers
  6. Fill the details with your CosmosDB collection information.
  7. After saving the function, go to the integrate section and click on a big plus under Outputs column on the top right. Function app
  8. Set it to the same collection as in the trigger- in my case that’s “Tweets”. Have a look at my settings. Function app triggers
  9. Save and go to the Function App Settings. Function app
  10. Insert your cognitive services key into the app settings – you’ll use it later in the function. Function app settings
  11. Now the fun part, we have to write the logic responsible for calculating the sentiment of the tweet.
  12. The code is not complicated – you need to iterate through the documents that came from CosmosDB, send a few requests to cognitive services with a tweets text then save it to output variable. Don’t want to write that code? Feel free to use the one in my github repo.
  13. If you use my code – after inserting all the three files, you have to go to: https://[YOUR_FUNCTION_NAME].scm.azurewebsites.net/DebugConsole and run npm install in the catalog containing package.json

Display tweets

Probably a bit hacky, but it works! Would you prefer to write another boring API?

  1. Go to the CosmosDB and find the section Add Azure Search.  Click on it and and click on create new search service. Azure search
    Azure search
  2. When you’re done – choose the newly created search service. Now go to the next section of creation wizard.Azure search source
  3. Choose your CosmosDB and your collection of tweets. Important note: make sure you have an existing document in the collection already (add some tweet and wait for it to get stored). Otherwise you will get an error, that the service cannot extract fields for the search index. Azure search index source
  4. After it’s done, choose the fields for the search index to contain. Azure search index fields
  5. Finally, the last step of the wizard is setting up the indexer (service that will insert data to the search index). Set it to run hourly.
  6. After you create it, remember to set the CORS options to every origin (or the origin of your app that will use it). Otherwise you will have problems with getting the mentions.
  7. Right on! Now, create the service. Wait a little and it’s done. Congratulations. you have the endpoint you can use to obtain the brand mentions with the sentiment calculation.
  8. If you want to display it in a nice way (like I showed you in the beginning) – you can create very simple website. Click on “Create a new resource again” and choose web app. I decided to create a really simple and dumb react app that will just display the list of mentions from Azure Search endpoint. I used the boilerplate from my dear friend – visit his repo. If you don’t want to mess with website development now, just use my code from github!
  9. The code of the website can be deployed in many ways – the fastest would be using Local Git repository (simply commit and push the code to the git endpoint created in Azure Portal) or just use the FTP (you have to setup deployments credentials in both cases). Azure deployment optionsDeployment credentials
  10. That’s it! By now you should have a working process of brand monitoring and a nice, working website displaying the mentions. Enjoy!

Conclusion

The cloud can be used to not only to solve the the big, enterprise problems – unique for a specific business. We can use it to just make our daily routine easier or cheaper.

Source: https://giphy.com/explore/smart

I started using that tool for brand monitoring of one of my projects on Twitter. It helps me to learn how to interact with Twitter people. Have a look at the result.

Maybe in time I will eventually migrate to some SaaS service. For larger scale it’s more cost effective and gives a lot more functionality. For now it’s sufficient enough to:

  • help me with a proper communication on one of the social media
  • monitor the reach of my brand presence
  • calculate my needs for a more expensive tool

Note: I could go down with the monthly price for the tool I created with replacing CosmosDB with some other database, but that would cost me a bit more time in creating the solution. Time worth more than a few euros per month. Feel free to experiment on your own!

How can I put it into practice?

If you want to start brand monitoring on Twitter for  a small money – you can grab all of the code I created in this proof of concept and use it or modify it.

Go to my github.

With my instructions and my code you should be able to create a working solution in no time!

Today I also showed you possibilities of a few cloud services. I showed you how it can bring benefit even in quite small areas of life. Think about the activities you spend time on. Maybe some of them can be easier, cheaper or less pain in the back? My example is just one way of using cloud in making business a bit less time/money consuming.

If you don’t need brand monitoring – you can try to adjust this solution to any other case. Maybe you’ll find some other problem fitting a similar process. If so – write a comment and let’s try to solve it together.

Most of all, I hope you see that it’s fun to experiment with the cloud capabilities and that we can create a solutions that could solve our existing problems in a cheaper and easier way.

Source: https://giphy.com/gifs/chemistry-KPn24x701Asus

See you around!

Series Navigation<< [Dev Cloud Essentials] How to never worry about your WordPress hosting?

Leave a Reply

Your email address will not be published. Required fields are marked *