Information retrieval and the challenge of scale

The revolution in Internet and Computer science has led to substantial improvement in the storage of information and its accessibility. Information retrieval has made libraries and search engines easily work for everyone, individuals and businesses. In my opinion, the two main advantages of information retrieval are firstly, providing users with relevant information they need from different resources, and secondly supporting them in making decisions on relevant issues by reducing information overload.

However, the issue I would raise here is the ability of these websites and search engines to continue to function well as they have limited size and volume. How these webs and search engines can face the increase of data volume, the diversity of the ways of information dispersion across the glob, and huge amount of information, visual and texts, from the daily news and the advancement of science and technology.

How to provide scalable performance for rapidly increasing data and workloads is critical in the design of next generation of information retrieval systems

Scalability is one of the problems that face computer applications and Internet websites by limiting their ability to provide users with continuing good services. Therefore, new methods and technology of computers, information retrieval and network must be merged to address the scalability concerns. Lets me give you a practical example, I mentioned in the previous post the project of my friend, which was to create a scientific network for Saudi students from different disciplines particularly who are studying in the United kingdom to discuss some important issues. In addition, two main search engines for jobs and courses were created in order to attract the students to join the network. However, one of the main problems of the project was the scalability, and as the developer said the solution for this problem might be either using lots of hardware resources, which cost money, or improving the design of the information retrieval systems to face the rapid increase in data and information.

Quote | This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s