Signal to Noise Ratio on Internet and solving problems

In Radio operations we use SN Ratio as one of the measures of quality of signal reception. better the SN ratio quality of signal is of better quality. When signals degrade under heavy noise we use various analog and Digital filters to dig signals out of noise and make it readable.

Today with twitter, facebook, blogs, RSS feeds etc., world is full of internet Noise. If one is just reading internet, soon (i mean pretty soon) he/she will get overwhelmed. Efficient filters and focus on real meat is very important. One such filtering product i recently reviewed was Infostreams . Its a product which can be used to get filtered information from RSS feeds/blogs.

I can see that in near future, even the digests, filters can become overloaded. Advanced Visualization technologies like word clouds, relational maps, heat maps may become prominent.
For example when you answer "what are you doing?" question, people who are having similar thoughts might be presented. Heat maps on geographies, topics etc could present massively data mined information. Identification of patterns and discoveries may take place on public collaborative platforms..

Enjoy !

Data Masking: What and why?

Trigger for this blog: IBM released their version of data masking solution http://www-03.ibm.com/press/us/en/pressrelease/22209.wss

This is not new thing i am blogging here. Masking mostly (notoriously) known in IT and widely/strictly adhered process in any software development practice. Security teams scan your systems, keep tab on support teams checking what they do with company's wealth.

Its all about dealing with production data in non production environments. Lovely, says my friend in testing group. Ok, this is very dangerous thing any organization can get into. Knowingly or unknowingly playing(so called organized testing) with production data can get companies in deep troubles of lawsuits and risk of getting out of business as well. In other words its also law in most of the countries keep customer and financial information etc secured and tightly controlled. Hmm... This is good but most of development team is going to be unhappy, because they will not get production data in test environment.(because they love to test on production data, or lazy to produce good test data)

Wait.. there is a solution >> Data Masking. (one of my friend calls these terms as 'old wine in new bottle': people used to do it before and now its named different)

So what is it?
Data masking is a systematic process where in Real data is masked in transformation stage so that when data is persisted in Test region, the data values would be scrambled.

Simple?! no. not really >> real challenge is assembling multiple such data elements and relating them. i.e for example Kiran has SSN as 123-45-6789 and if its scrambled to 987-65-4321 (scrambling is never this straight forward though), and if there is a requirement to link another data set of Kiran by SSN, if both datasets were not scrambled using same technique, then we would not be able to assemble and make any sense. This is real challenge in most of data warehousing environments today.

There are many product companies addressing this problem today.

Personally, i am interested to see how these solutions help companies to balance the internal quest for information mining without breaching security.