Page 1 of 1

Saving entire threads

Posted: 2010-02-21 01:27am
by eyl
I've occasionally (in this forum and others) run across threads which were interesting/amusing/etc. enough for me to want to save offline. If the thread contains only a single page, that's simple - I can just print to PDF. If there are a few pages, no biggie - print to PDF and just merge the files, or else copy/paste everything to Word. However, this gets problematical if we're talking about a 30-page thread.

Does anyone know of software which would allow me to pull offline an entire thread? I tried to use PageZipper for this, but it tends to pull up pages from the next thread as well, which I don't want.

Re: Saving entire threads

Posted: 2010-02-21 03:06am
by Shadow6
There are a few different applications/addons which can manage this for you. If you're running Firefox, my two favourites are the ScrapBook extension (offical addon page, author's page) and the FlashGot extension partnered with your preferred download manager (offical addon page, author's page).

On SD.net, the threads have a URL of the form:

Code: Select all

http://bbs.stardestroyer.net/viewtopic.php?f=<FORUM ID>&t=<TOPIC ID>&start=<REPLY NUMBER AT TOP OF PAGE>
Essentially, you tell which ever addon you chose to iterate through the pages of the thread by incrementing the page number by 25 each time. For instance, to download the entirety of this thread using FlashGot, you would input the URL as:

Code: Select all

http://bbs.stardestroyer.net/viewtopic.php?f=2&t=140525&start=[0-125;25]
If you're competent with Bash/Command Prompt, then there are a variety of utilities, such as wget. Plenty of guides exist around the net on how to use wget, so I'll leave that to you.

Hope that helps and I'm happy to provide further assistance if necessary.

Re: Saving entire threads

Posted: 2010-02-21 10:54am
by [R_H]
I've been wondering how to do this as well for some time now, but until now I've just been copy/pasting into a text document. Thanks Shadow6.

Re: Saving entire threads

Posted: 2010-02-21 01:55pm
by eyl
Is there a tutorial available for FlashGot anywhere? I tried to download the thread you gave as an example (using FlashGot's Build Gallery) but it doesn't seem to do anything. Is there another way to do it?

Re: Saving entire threads

Posted: 2010-02-21 02:50pm
by Shadow6
eyl wrote:Is there a tutorial available for FlashGot anywhere? I tried to download the thread you gave as an example (using FlashGot's Build Gallery) but it doesn't seem to do anything. Is there another way to do it?
Hmm, I probably should have mentioned a specific download manager in my post. I personally like JDownloader. The process is pretty simple, you just use the Build Gallery function as you did and drag the resultant links onto the JDownloader window.

However, I have stumbled upon another extension for Firefox which simplifies the matter somewhat. DownThemAll! (offical addon page, Author's Page) is yet another batch downloader, but features its own download manager and hence can compile and download the pages itself. In a similar process to FlashGot, you open the Manager and add the URL as:

Code: Select all

http://bbs.stardestroyer.net/viewtopic.php?f=2&t=140525&start=[0:125:25]
Because all of the files are named viewtopic.php, you'll need to instruct DownThemAll! to rename each file. You can then do what you wish with the resultant files.

Re: Saving entire threads

Posted: 2010-02-23 12:20pm
by eyl
Nice. I think I'll try a little VBA scripting and see if I can automate the process (and merge the pages, while I'm at it)