Saving entire threads

GEC: Discuss gaming, computers and electronics and venture into the bizarre world of STGODs.

Moderator: Thanas

Post Reply
eyl
Jedi Knight
Posts: 714
Joined: 2007-01-30 11:03am
Location: City of Gold and Iron

Saving entire threads

Post by eyl »

I've occasionally (in this forum and others) run across threads which were interesting/amusing/etc. enough for me to want to save offline. If the thread contains only a single page, that's simple - I can just print to PDF. If there are a few pages, no biggie - print to PDF and just merge the files, or else copy/paste everything to Word. However, this gets problematical if we're talking about a 30-page thread.

Does anyone know of software which would allow me to pull offline an entire thread? I tried to use PageZipper for this, but it tends to pull up pages from the next thread as well, which I don't want.
Shadow6
Youngling
Posts: 146
Joined: 2009-01-20 08:58pm

Re: Saving entire threads

Post by Shadow6 »

There are a few different applications/addons which can manage this for you. If you're running Firefox, my two favourites are the ScrapBook extension (offical addon page, author's page) and the FlashGot extension partnered with your preferred download manager (offical addon page, author's page).

On SD.net, the threads have a URL of the form:

Code: Select all

http://bbs.stardestroyer.net/viewtopic.php?f=<FORUM ID>&t=<TOPIC ID>&start=<REPLY NUMBER AT TOP OF PAGE>
Essentially, you tell which ever addon you chose to iterate through the pages of the thread by incrementing the page number by 25 each time. For instance, to download the entirety of this thread using FlashGot, you would input the URL as:

Code: Select all

http://bbs.stardestroyer.net/viewtopic.php?f=2&t=140525&start=[0-125;25]
If you're competent with Bash/Command Prompt, then there are a variety of utilities, such as wget. Plenty of guides exist around the net on how to use wget, so I'll leave that to you.

Hope that helps and I'm happy to provide further assistance if necessary.
[R_H]
Sith Devotee
Posts: 2894
Joined: 2007-08-24 08:51am
Location: Europe

Re: Saving entire threads

Post by [R_H] »

I've been wondering how to do this as well for some time now, but until now I've just been copy/pasting into a text document. Thanks Shadow6.
eyl
Jedi Knight
Posts: 714
Joined: 2007-01-30 11:03am
Location: City of Gold and Iron

Re: Saving entire threads

Post by eyl »

Is there a tutorial available for FlashGot anywhere? I tried to download the thread you gave as an example (using FlashGot's Build Gallery) but it doesn't seem to do anything. Is there another way to do it?
Shadow6
Youngling
Posts: 146
Joined: 2009-01-20 08:58pm

Re: Saving entire threads

Post by Shadow6 »

eyl wrote:Is there a tutorial available for FlashGot anywhere? I tried to download the thread you gave as an example (using FlashGot's Build Gallery) but it doesn't seem to do anything. Is there another way to do it?
Hmm, I probably should have mentioned a specific download manager in my post. I personally like JDownloader. The process is pretty simple, you just use the Build Gallery function as you did and drag the resultant links onto the JDownloader window.

However, I have stumbled upon another extension for Firefox which simplifies the matter somewhat. DownThemAll! (offical addon page, Author's Page) is yet another batch downloader, but features its own download manager and hence can compile and download the pages itself. In a similar process to FlashGot, you open the Manager and add the URL as:

Code: Select all

http://bbs.stardestroyer.net/viewtopic.php?f=2&t=140525&start=[0:125:25]
Because all of the files are named viewtopic.php, you'll need to instruct DownThemAll! to rename each file. You can then do what you wish with the resultant files.
eyl
Jedi Knight
Posts: 714
Joined: 2007-01-30 11:03am
Location: City of Gold and Iron

Re: Saving entire threads

Post by eyl »

Nice. I think I'll try a little VBA scripting and see if I can automate the process (and merge the pages, while I'm at it)
Post Reply