Distributing large textures really fast

[selden]

I contacted LoCI directly.

While their current policy does permit public use of the service, individual institutions may decide to block access from the public Internet.

This is because most computer science departments have to pay for their network bandwidth usage. For example, Cornell has a cost recovery procedure called "Network Usage Based Billing" and charges departments $2 per gigabyte of traffic to or from the commercial Internet.

[rich]

The main problem for one of us who wishes to share a large texture is that a lot of storage space and bandwidth is required for say ~50 people to download that texture. A member of this forum (Jestr) is sharing large textures via ftp. But I only get 10kb/s on a 64kb/s ADSL connection so it can take days to download!

Using LORS there are no bandwidth costs or storage problems. And everyone with cable modems can download at the maximum speed of their connection (64kb/s, 128kb/s, etc.) without overloading the server. As most people will be doing this from home connections there shouldnt be a problem with violating university policies.

LORS also offers linux distributions for download which is hardly of crucial scientific importance either.

[selden]

Scott Atchley of LoCI gave me permission to quote his response to my query.

On Apr 16, 2004, at 3:50 PM, Selden E Ball Jr wrote:

> Gentle folk,
>
> I have not been able to find any statement on the LoCI Web site which
> explicitly states who may use the LoCI file services.
>
> Is it open to the public or is it is limited to Internet2 subscribers?
>
> Thanks for whatever clarification you can provide.
>
> Selden

Date: Fri, 16 Apr 2004 15:59:44 -0400
From: Scott Atchley
Subject: Who may use LoCI file services?
To: Selden E Ball Jr

Hi Selden,

We have started a new list for general questions regarding LoCI. I have
copied the list on this message.

The depots are currently available for all to use. We rely on the many
institutions that host the servers. If the amount of traffic to their
commercial Internet (non-Internet2) connections grows too high (many
pay for the volume transferred), they may opt to restrict the traffic
to Internet2 or, worse, request the depot be turned off.

Given that, feel free to use the public servers. You can also add your
own resources very easily (in under 15 minutes). You can then choose to
make your resources public as well or you can run a private network. We
encourage people to do both.

Scott

and

Date: Sat, 17 Apr 2004 08:31:32 -0400
From: Scott Atchley
Subject: Re: Who may use LoCI file services?
To: Selden E Ball Jr

Hi Selden,

You may quote me if you like. If you do, please also include the
following:

We only ask that users have legitimate rights to the data that they
store and transfer.

Good luck. If you or your fellow users have questions about using our
tools, please join loci-interest__cs_utk_edu. If you post that address,
try to make it not look like an email address.

Thanks,

Scott

(I replaced @ by __ and . by _ )

[t00fri]

The main problem for one of us who wishes to share a large texture is that a lot of storage space and bandwidth is required for say ~50 people to download that texture. A member of this forum (Jestr) is sharing large textures via ftp. But I only get 10kb/s on a 64kb/s ADSL connection so it can take days to download!

Using LORS there are no bandwidth costs or storage problems. And everyone with cable modems can download at the maximum speed of their connection (64kb/s, 128kb/s, etc.) without overloading the server. As most people will be doing this from home connections there shouldnt be a problem with violating university policies.
...

Rich,

yes, I agree with this. With a cable modem you could expect to reach the maximum theoretical download speed, but the real fun starts when you download in multithreaded mode with 100+Mb/sec!

Scientists (like myself) do have access in their offices to
fast research networks, but are strictly forbidden to use them for non-pro purposes...too bad.

I was somewhat mislead by the title of your thread, where you advocate "really fast" downloads for big textures;-). 64kb/sec cable speed is not what I call "really fast". That's why I anticipated misuse of "really fast research networks"...


Bye Fridger

[John Van Vliet]

well i am felling adventures so i will give it a look an a try
but it's 4:30 am so it will have to wait until tomarow
by the way would the
Computer Science Department of the University of Tennessee
use post nuke for a web site
that is what mozila says it is , in the page info window

[Star Lion]

Wouldent using a file sharing network like bitorent work even better?
The first few downloads would be slow, but after the network became saturated it would be no big deal

[rich]

bittorent is not a good solution because a) download speed often only reaches 20kb/s on a 64kb/s line. and b) is not very permanent i.e. after a few weeks it may not work as nobody is "seeding" the file. It requires a number of people who have the complete file to be running the bittorrent program constantly.

[greenwood]

If anyone has problems using Lors, contact me via email for help. I've been using this for a while now, a big advantage is the ability to move large files to servers that are local to you before downloading.

[Psykotik]

Well, emule network is MADE for big files, and to STORE big files. And I have to say that's going very well, although not fast enough... The more people will be involved into, the faster it will be !

I'm sharing 230 files using this network, and for people concerned by children protection, you can download a protected version (ouioui, Christophe

[rich]

psykotik - the great thing about LORS is it doesn't matter whether you have 1 sharer or 500 sharers you still download at the maximum possible speed for your connection ALL THE TIME. emule is very unlikely to give you maximum speed constantly. so why do you think emule is better?

[greenwood]

Earth VT now available on LORS in JPG format (258Mb)

XND file is available from http://askuk.net/jpgmarble.zip.xnd

Earth VT levels 0-10 includes southern England, parts of Canada, Comahue, Corsica, Cyprus, Boston, New York, Hawaii, Venice, middle east, Aercibo and various Islands.

Credit goes to Jestr, kinderino, Guillermo Abramson and jim

[rich]

i downloaded your xnd but it doesn't work!

the lors client map shows 2 active connections for about 5 seconds then it goes to 0 (nothing is downloaded).

also this doesn't seem to be zip compressed so why is there a zip extension!

thanks

[greenwood]

i downloaded your xnd but it doesn't work!

the lors client map shows 2 active connections for about 5 seconds then it goes to 0 (nothing is downloaded).

also this doesn't seem to be zip compressed so why is there a zip extension!

thanks

I think the file expired (the default was set to 1 day). I've uploaded it again but won't have time to test download till Friday. You'll need to reget the xnd file as I've changed the filename.

If you can wait till Friday I'll post here after testing the download

kind regards
Karl

[transman]

LoCI Lab is testing a new service called the Logistical Distribution Network (LoDN, pronounced "lowdown") which is a directory for maintaining exNodes indefinitely and creating an environment for manipulating them. One can apply for an account by going to this URL

http://promise.sinrg.cs.utk.edu/lodn

and clicking on the link that says "Click here to sign up." You will be asked to provide a brief description of your application - "distributing Celestia textures" is adequate.

Some of the new features available through LoDN are a Java upload and download, which obviate the need to install the LoRS tools on the users' desktop and a publication mechanism that allows a read-only version of an exNode to be published, obviating the use of e-mail or a private Web server to distribute exNodes, and providing protection of the published data.

The service is not yet announced, the interface is still under development, and there is no documentation yet. However, it's not too hard to figure out, help is available from the loci-interest@cs.utk.edu mailing list, and it works. LoDN is invaluable for anyone doing content distribution using exNodes.

Of course, LoDN must not be used for distirbution of copyrighted or other inappropriate content.

[Wbdsgnr]

I use linux, and I tried downloading the file using this command:

/usr/bin/lors_download -f -C 10 -t 10 -b 512k jpgmarble.zip.xnd

ofcourse I first compiled lors. It starts downloading until the file is 5242880 bytes... then it stops, is it me? I tried the command without '-b 512k' but that doesn't help. Did anyone succeed downloading the file?

[transman]

That exNode has holes in it, probably due to the fact that

1. only one copy of the data was stored in the Logistical Network, and
2. some of the data allocations have become unavailable or expired.

Managing data using exNodes is not that hard, but it takes a little bit of getting used to. Luckily, there is some documentation:

http://loci.cs.utk.edu/modules.php?name ... it&lid=184

The aforementioned LoDN service makes things much easier:

http://promise.sinrg.cs.utk.edu/lodn