Welcome, Guest. Please login or register.

Login with username, password and session length

 
Advanced search

1411196 Posts in 69314 Topics- by 58380 Members - Latest Member: feakk

March 19, 2024, 01:16:11 AM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsCommunityJams & EventsCompetitionsOld CompetitionsAssemblee: Part 1Assembleech - The Assemblee Link Leech (19/12 update)
Pages: [1] 2 3 4
Print
Author Topic: Assembleech - The Assemblee Link Leech (19/12 update)  (Read 38412 times)
Sos
Level 8
***


I make bad games


View Profile WWW
« on: November 19, 2009, 07:26:07 AM »

I did some PHP today and made a script that runs through all the posts in Assemblee Part 1 subforum and gathers all the links that do not point to tigsource.com or directories.

The output of this script can be found here:



The Assembleech


Last Update: Saturday 19th of December 2009 19:33:05 GMT
List weight: 631.23 kb (646 375 bytes)
Fetched 4049 posts in 380 pages containing 4778 great things.


I will update this manually on a regular basis to avoid DDoSing the tig. It does contain some garbage too, like rapidshare-like links and maybe some others too, but download managers can filter the content you wish to acquire so i think it's not much of a problem.

Hope this is helpful.

I managed to add a simple search function which can be found here:


It searches for anything within link's URL, and returns thread from which it originated, and author's name (will return UNKNOWN_AUTHOR for links found in stickies)


Also, thanks to nitram_cero we have a tool, that downloads files to separate folders, according to the author. Then this can be used with Indie Asset Organizer.
Indie Asset Organizer
by nitram_cero

Cheers,
Sos

PS

My server keeps getting a strange "Internal Server Error" When trying to run the script, I think I expect too much of it, and it actually feels bad inside. So I enabled everybody to donate something nice to the server, to make it feel better.


My server seems to like: liquor, food, flowers, meadows, endangered species, firearms and other nice things.

I already donated my favourite vodka Smiley
« Last Edit: December 19, 2009, 11:43:47 AM by Sos » Logged

Derek
Bastich
Administrator
Level 10
******



View Profile WWW
« Reply #1 on: November 19, 2009, 08:06:30 AM »

That's awesome!

 Hand Thumbs Up Left Hand Thumbs Up Left Hand Thumbs Up Left
Logged
brog
Level 7
**



View Profile WWW
« Reply #2 on: November 19, 2009, 08:09:16 AM »

So the Assembleech will eventually find itself and link recursively?
Logged
moi
Level 10
*****


DILF SANTA


View Profile WWW
« Reply #3 on: November 19, 2009, 08:22:59 AM »

Man, the assemblee will soon take over the internet
Logged

subsystems   subsystems   subsystems
Sos
Level 8
***


I make bad games


View Profile WWW
« Reply #4 on: November 19, 2009, 03:10:09 PM »

Ok, I leeched supposedtly everything that the script found in the forums, and made some statistics (just for fun).

I managed to download 1529 files that weigh 507MB
74MB (1428 files)
are graphics ranging from a giant watermelon to an animated fat guy that melts to pixels.
101M (13m07s) are the sound effects  ranging from a beep to voice acting of "om nom nom"
295MB (3h52m27s) is the music ranging from live harmonica to some chiptune modules and it is my current playlist (this is just great)

60MB
are other things extracted from archives: dirs, models, indie asset organisers and readmes which are pretty useful

That would be it.

Smiley

Question
: how many unique games can you make out of that?
« Last Edit: November 19, 2009, 03:25:57 PM by Sos » Logged

J. R. Hill
Level 10
*****

hi


View Profile WWW
« Reply #5 on: November 19, 2009, 04:40:55 PM »

it is my current playlist (this is just great)
Haha wait till you get to my loops and you get about 1 min of random sounds.  Durr...?
Logged

hi
Martin 2BAM
Level 10
*****


@iam2bam


View Profile WWW
« Reply #6 on: November 20, 2009, 04:14:46 PM »

This is awesome!

Do you think you can extract the original poster username?
It would be nice to for crediting

Or at least the topic URL to know where they came from.

Regards
Logged

Working on HeliBrawl
Sos
Level 8
***


I make bad games


View Profile WWW
« Reply #7 on: November 22, 2009, 01:46:17 PM »

UPDATED TODAY!!!

I tbhought about categorising the links, but:
- Download managers such as downthemall can easily download JUST the desired content.
- Categorising by author is done by the forums

But I think I can come up with something to aid that.

Logged

Martin 2BAM
Level 10
*****


@iam2bam


View Profile WWW
« Reply #8 on: November 22, 2009, 01:56:22 PM »

I don't mean to be a douche, but a lot of files will not be directly downloadable because they're hosted in free servers that need human interaction (CAPTCHA) to start the download.

And I was asking for the auther to possible integrate it easily with this Well, hello there!

Best regards!

BTW: Great stats gathering Wink!
« Last Edit: November 22, 2009, 02:00:18 PM by nitram_cero » Logged

Working on HeliBrawl
Sos
Level 8
***


I make bad games


View Profile WWW
« Reply #9 on: November 22, 2009, 02:49:38 PM »

I dunno if i could be able to integrate it with Indie Asset organiser, but I added the lookup functionality you was asking for. You can lookup a filename and it will return matching links and lionk to page that they were found on, so if a pic was posted on 3rd page of a thread, it will link to this page directly. And yes, many people use this sharing sites, but I will not be able to work this one around.
Logged

Captain_404
Guest
« Reply #10 on: November 22, 2009, 03:19:07 PM »

Could you set it up to look for common sharing site names in the url?

EDIT: wait... I think I misunderstood the problem...
« Last Edit: November 22, 2009, 03:24:21 PM by Captain_404 » Logged
Sos
Level 8
***


I make bad games


View Profile WWW
« Reply #11 on: November 22, 2009, 03:22:46 PM »

Could you set it up to look for common sharing site names in the url?
It does that, see for yourself: http://so0os.pl/assembleech/search.php?search=mediafire

It looks for anything that's in the url.
Logged

Captain_404
Guest
« Reply #12 on: November 22, 2009, 04:40:07 PM »

Could you set it up to look for common sharing site names in the url?
It does that, see for yourself: http://so0os.pl/assembleech/search.php?search=mediafire

It looks for anything that's in the url.

Yeah, I misread your post, sorry for the confusion!
Logged
Skab
Level 0
***



View Profile WWW
« Reply #13 on: November 23, 2009, 02:43:18 AM »

very nice an usefull tool  Cool
Logged

Sos
Level 8
***


I make bad games


View Profile WWW
« Reply #14 on: November 23, 2009, 02:20:26 PM »

Just updated.

I thought about it, and I won't be making keyword search, just because the forums has one. Enjoy
Logged

Tesse
Level 0
**



View Profile
« Reply #15 on: November 23, 2009, 02:27:00 PM »

So cool !  Hand Thumbs Up Left Hand Thumbs Up Left Hand Thumbs Up Left
And amazing to see THAT much content !! Addicted
Logged

Please be friend of me.
Sos
Level 8
***


I make bad games


View Profile WWW
« Reply #16 on: November 23, 2009, 02:32:26 PM »

And amazing to see THAT much content !!

I hope the games will be as numerous as that Smiley
Logged

Sos
Level 8
***


I make bad games


View Profile WWW
« Reply #17 on: November 25, 2009, 10:37:54 AM »

Just an ordinary update bump. (Updated just now)

Sorry i don't update often but the script is really heavy. Sometimes it works after 3-4 tries (returns "500 - Internal Server Eror, please contact the administrator at [email protected]")
Logged

Melly
Level 10
*****


This is how being from "da hood" is like, right?


View Profile
« Reply #18 on: November 25, 2009, 10:44:35 AM »

Sexylicious. Stickied, for great justice.
Logged

Feel free to disregard the above.
Games: Minus / Action Escape Kitty
Sos
Level 8
***


I make bad games


View Profile WWW
« Reply #19 on: November 25, 2009, 01:36:56 PM »

Thanks Melly Smiley

That'll save some useless bumping
Logged

Pages: [1] 2 3 4
Print
Jump to:  

Theme orange-lt created by panic