Re: [squid-users] Jobs

On Wed, 2011-06-29 at 21:04 +1200, Amos Jeffries wrote:
> On 29/06/11 19:28, Mohsen Pahlevanzadeh wrote:
> > On Wed, 2011-06-29 at 02:32 +1200, Amos Jeffries wrote:
> >> On 29/06/11 01:37, Mohsen Pahlevanzadeh wrote:
> >>> On Tue, 2011-06-28 at 06:01 -0700, John Doe wrote:
> >>>> From: Mohsen Pahlevanzadeh<mohsen_at_pahlevanzadeh.org>
> >>>>
> >>>>>>> We must write a program that along with normal tasks, it had do a
> >>>>>>> variety of jobs,But i need to PURGE and insert cache.
> >>>>> I must a write a web appl thet it manages squid with many extra job.
> >>>>> normal job: every work that squid can do.
> >>>>> Variety of job: a range of task that my boss ordered.
> >>>>
> >>>> Amos question: What are these "normal tasks" and "variety of jobs"?
> >>>> Your answer: extra job, normal job, a variety of job, a range of tasks...
> >>>> Which does not answer the question at all...
> >>>> Can you name the main tasks/jobs you need to do?
> >>>> By example: start/stop/restart/reload squid, reset cache, purge/cache url?
> >>>> Graph statistics, etc...
> >>>> I believe that for most of these, you do not need to play with the squid code...
> >>>>
> >>>> JD
> >>> 1. PURGE from my program, but i can't call squidclient PURGE -m
> >>> "blahblah" from my code.
> >>
> >> Sure you can. Several prefetchers just run exec("squidclient blah")
> >>
> >>
> >> But now that it is clear you are building a whole management app, not
> >> just a prefetcher an HTTP library would probably be the better way to
> >> go. libcurl or whatever the equivalent is for your chosen language.
> > It has problem. Because you suppose server has 2000 or more request
> > concurrent, Then 2000 times squidclinet had run, it's wrong......
> > So i must write a function same as purge and do it from code.
> >>
> >>> 2.Insert into cache.
> >>
> >> HTTP GET request. Tricky. Since you will have to figure out whether the
> >> clients will be asking for plain or compressed copies.
> >>
> >> By far and away the best way to do this is simply not to bother doing it
> >> at all. Squid is designed to do the work of figuring out where objects
> >> are and how to get them to the client fastest.
> >>
> >> Inserting objects into the cache may _seem_ to be a good idea. But HTTP
> >> is very complicated and there is a very good chance you will push the
> >> wrong variants of each object into the cache.
> >>
> > Your suggestion is very very nisce and i'll start HTTP Definitive Guide
> > of O'Reilly.I know libcurl, it's good idea that we request to squid for
> > push and squid itself push.
>
> My point earlier was to use libcurl for PURGE as well. No difference to
> you between PURGE and GET other than the name.
>
> >
> >>> 3.concurrent receive of site(minimum 100 sites)
> >>
> >> If by "site" you mean website. Squid is used by ISP. They have
> >> accessible site numbers ranging in the high millions or billions. These
> >> are all concurrently available to an ISP situation, so safe bet on that
> >> requirement.
> >>
> >> If by "site" you mean visitor. One Squid routinely handles hundreds or
> >> thousands of clients depending on your hardware specs. Or it may
> >> overload the network on _one_ client requesting TB sized objects.
> >>
> >> You need to figure out a request/time-unit metric or a concurrent
> >> connections metric and test that is achievable with the desired
> >> configuration. The squid config file is a mix between simple
> >> on/off/value settings and a big script which tells Squid how to operate
> >> on a request. Seemingly simple changes can easily raise or lower the
> >> response speed by whole orders of magnitude.
> >>
> > We must do following task:
> > 1. Possibility to download concurrent 100 sites (website) from
> > internet.
> > 2.Possibility to filter them that i know squid uses url_regex.
> > 3.Possibility to answer as proxy for minimum 1000 concurrent request.
> > 4.Time of answering to each request must be 100 mili second.
>
> Possible trouble.
>
> If you were setting up a reverse-proxy situation with controlled
> network between the proxy and web servers that could be a realistic
> goal. In those cases it is relatively easy to get a lot of requests in
> the low dozens of millisec.
>
> However your earlier requirement (1), indicates that your traffic will
> be from the Internet. You will be 100% at the mercy of external
> administrators when it comes to timing.
>
> > 5.Space and algorithm should be chosen that can expand to store 10000000
> > pages.
>
> Sounds like you are tasked with creating a whole new proxy system. That
> was all tasks _your app_ has to do was it not?
>
> Amos
You suppose i have a server A and its a logger and server B is
squid.Server A is connected to the internet.Server A download pages and
server wants to deliver pages to squid:3128 but i don't know squid can
accept just pages?How i do instruct it to server B?
--mohsen