Fwd: using squid for distributed cache

HI,

I think that squid-dev mailing list is a better place to post your
request, so I'have forwarded your message to the list.

Regards

Guido

At 01:36 25/03/2008, Hasanat Kazmi wrote:

>Hello,
>I am Hasanat Kazmi, Our team of students are doing a research
>project (called Poor Man's Broadband). We want to make a system to
>boost internet speed for third world countries where internet speed
>is really slow and people still rely on dial up modems. We want to
>deploy a modified version of Squid for our proposed system and I
>don't think there would be some one else who knows Squid better than you.
>
> Internet infrastructure in our countries are also not that good. I
> hail form Pakistan, so I can give a view of what we call back bone
> of internet infrastructure:
>Unlike US, in Pakistan we have more than 200 ISPs in different
>towns. On average these ISPs are not providing services to more than
>2000 people or so. Moreover their cashing system is also very weak,
>and unlike US, there is no concept of central cashing in ISPs at all.
>Majority of the servers connected to the internet reside in US. The
>maximum internet traffic from Pakistan flows to US and if we
>analyze, more than 60% of this traffic is going to few popular websites.
>Now the problem by example:
>Suppose, I am node A and I want to download a video lecture from
>MIT's open course ware. Now, Ill spend some good 3 hours to download
>30 minutes lecture. Suppose node B also wants that lecture and he
>will also spend same time no matter if he is just sitting in my
>neighborhood and is connected through some other ISP . Here we have
>two problems:
>1) There is no central cache
>2) Internet speed is really slow
>And here is what we want to do,
>Internet speed is too slow but we are two people who want this file,
>lets suppose I have a service called ABC. what ABC does is that it
>receives the request of the file to be downloaded (yes,both nodes
>will ask me to help them with the file). Now what Ill do is that Ill
>see that how many people want this file. If the file is popular,
>there will be many requests, say i have 10 requests, then I divide
>the file to be downloaded into 10 chunks and assign each user (by
>using the software they requested ABC) one chunk to download, now
>each client/node will take 3/10 hours to download this file. But after this :
>
>recommendation 1:
>Server Based: They will upload that chuck to ABC's servers. Now ABC
>is hosted inside the country and is more accessible. They can upload
>the there parts in may be 10 minutes. Now, on the server, i join
>these chunks to make a working video file, and then I start sending
>this file to all those who requested me for the file. Again, as I am
>not in US but I am locally hosted, The file transfer speed will be
>times faster than that of downloading form MIT's servers. In this
>way each node may get there required file in may be less than 45 minutes.
>Moreover, if a file is popular, i.e ABC is getting about 200
>requests to download that file a day, then rather than deleting the
>file after transferring to the nodes, it may be kept in ABC's cache
>so that nodes dont spend time and resources downloading and uploading the file
>recommendation 2:
>P2P: We may have a node allocating and coordinating server. What it
>does is quite simple, it sees how may people want the file, then ABC
>(a different one of course) allocates each node a chuck to
>download. When they download the chuck (with slow speed from US),
>they start 'seeping' it to the other people who also wanted this
>files. (Wait this isn't bit torrent) And as other people are in
>nearby of the person who requested the file, it will have much
>better speed. In this case, our system can be scale to the whole
>word. We can make certain Regions 'Hot' for a file depending on the
>demand coming form that region for a specific file. Then we can make
>a small sub-system within that area, so that people can make use of
>this system locally. The boundries of this region are flexible and
>depend on the demand coming form users. Moreover, the region is
>different for each file requested. This system can be globally
>implemented using lesser resources.
>This System also creates cache like system, as all files are
>sweeping from local hood.
>
>Both these recommendation seems to overcome the two problems discusses above.
>
>
>A lot of academic work and research has been done on this system.
> Please visit
> <http://newt.lums.edu.pk/index.php/projects/poor-mans-broadband>http://newt.lums.edu.pk/index.php/projects/poor-mans-broadband
>and see what is has happened on this proposal.
>
>We want to use Squid and/or some other system which can help us
>achieve our goals. I will be very thankful to you if you can help us
>by telling us the limitations and plus points of squid for this
>system. Moreover, It will be really great if you can join the wiki
>we have developed for this project. (If you want to join, which I
>hope so, then please let me know so that I can send you invites) If
>you have some difficulties in understanding the system, then please
>let me know, Ill try to understand the things to you in alternate ways.
>
>Ill be waiting for you reply.
>
>
>
><mailto:10020513@lums.edu.pk>10020513@lums.edu.pk
>+92346362473
>

-
========================================================
Guido Serassio
Acme Consulting S.r.l. - Microsoft Certified Partner
Via Lucia Savarino, 1 10098 - Rivoli (TO) - ITALY
Tel. : +39.011.9530135 Fax. : +39.011.9781115
Email: guido.serassio@acmeconsulting.it
WWW: http://www.acmeconsulting.it/
Received on Sun Mar 30 2008 - 10:33:20 MDT