00:00
00:00
Newgrounds Background Image Theme

psychicpxl just joined the crew!

We need you on the team, too.

Support Newgrounds and get tons of perks for just $2.99!

Create a Free Account and then..

Become a Supporter!

Nglister - Ng Profile Data Miner

11,305 Views | 141 Replies
New Topic Respond to this Topic

Response to Nglister - Ng Profile Data Miner 2004-08-27 19:23:10


At 8/27/04 07:18 PM, D0GMA wrote:
At 8/27/04 06:57 PM, ShittyKitty wrote:
Cool stuff. Glad to see it's behaving itself ;-)

78.7% complete...
See my post here before you do anything with that data. I've resisted the suggestions that I ask liljim fill my blanks in for me for similar reasons ... I didn't want to establish a precedent that he would do so.

I know myself well enough to know that some of my feelings on the matter come from being wary of others "stepping on my turf," but it doesn't detract from the fact that I've got some valid concerns. The same kind of mindset that is willing to hunt through profiles to compile IDs for a list, and go through the ongoing troubles of maintaining it, is the same mindset that is going to ask, "is there even a need for the list?"

Perhaps I could re-build 1.3 with a password function? A list-builder would request a password from me to run the program within a certain time-frame, and I would generate the password and email it to them. The password would only work during that time-frame. Plus, by only distributing passwords to known list-makers (the ones that worry about too many lists being made), maybe this should help fix the problem. What do you think?

Response to Nglister - Ng Profile Data Miner 2004-08-27 19:35:44


At 8/27/04 07:23 PM, ShittyKitty wrote: Perhaps I could re-build 1.3 with a password function? A list-builder would request a password from me to run the program within a certain time-frame, and I would generate the password and email it to them. The password would only work during that time-frame. Plus, by only distributing passwords to known list-makers (the ones that worry about too many lists being made), maybe this should help fix the problem. What do you think?

NOTICE
I've taken NGLister off my website until further notice. I'll be developing a password protection solution to help limit the number of lists on Newgrounds.

Response to Nglister - Ng Profile Data Miner 2004-08-27 19:54:07


At 8/27/04 07:23 PM, ShittyKitty wrote:
Perhaps I could re-build 1.3 with a password function? A list-builder would request a password from me to run the program within a certain time-frame, and I would generate the password and email it to them. The password would only work during that time-frame. Plus, by only distributing passwords to known list-makers (the ones that worry about too many lists being made), maybe this should help fix the problem. What do you think?

Several versions (passwordless) are out there, so I don't see much need or effectiveness in doing so. It's not a problem now ... I'm just thinking ahead and taking into consideration the mentalities into which we constantly run here.

The same fucknuts that generate and submit flash trying to copy an idea (Battlefield/Quiz/B/Seizure/etc.) are the same kind of people who would make some bullshit list that shouldn't have been made in the first place. I'm always going to ask the question, "is there a need for the list," even on the EGRL. If there is a need, by all means make it, but these people won't even stop to ask ... they'll put up some crap, then complain that they shouldn't have gotten locked/banned since, "those lists are allowed." Another type of person will have every good intention when they make a list ... there's just no need for it to exist.

There's a myriad of lists that can be made. Personally, I don't post or review a lot, so I could see a need for those lists. I still don't see a need for the Pen-10-K, but others do, and that satisfies my question. The same goes with your idea for that compilation list of Portal entries. I don't see a need for it, but others might ... so I'm going to toss that in the mix. The real question is for a list, once its parameters are decided upon, is should it be made? With the gamut of personalities here, there's far too many people who wouldn't even stop to ask that question and I've a real concern that crap will flood the forum, to the detriment of the established lists, and we may miss out on some good ideas in that morass.

Response to Nglister - Ng Profile Data Miner 2004-08-27 19:54:08


At 8/27/04 07:35 PM, ShittyKitty wrote: NOTICE
I've taken NGLister off my website until further notice. I'll be developing a password protection solution to help limit the number of lists on Newgrounds.

Heh...now you make me feel like I have a pirated copy :)

I was a little afraid of this happening. I mean, it's an incredible breakthrough for these listmakers, but I can see that there would (potentially) be a surge of this happening and the NG bandwidth would suffer for the rest of the site.

I was predicting right after you created this, that it would eventually be one person maintaining ALL the lists on their site.

I could see d0gma making several lists on his site. One for XP, VP, so on...with this new NGlister, it wouldn't be too difficult for one person to do and maintain all the lists.

Don't think I am implying anything chris :P

Anyway, good job on halting this before it got out of control.

Response to Nglister - Ng Profile Data Miner 2004-08-27 20:05:26


At 8/27/04 07:54 PM, D0GMA wrote: Several versions (passwordless) are out there, so I don't see much need or effectiveness in doing so. It's not a problem now ... I'm just thinking ahead and taking into consideration the mentalities into which we constantly run here.

Yeah, but you guys (D0gma, RedCircle, and a few others) are the only ones who have downloaded the other versions. Now that this program has matured a little, and will start seeing more usage, its popularity should increase. Hopefully by providing a registration mechanism, I can limit the program's usage in the future. Yeah, I've decided to make the program registerable rather than making passwords. A reg code will only work on one machine, though; no need to worry about sharing of reg code's.

There's a myriad of lists that can be made. Personally, I don't post or review a lot, so I could see a need for those lists. I still don't see a need for the Pen-10-K, but others do, and that satisfies my question. The same goes with your idea for that compilation list of Portal entries. I don't see a need for it, but others might ... so I'm going to toss that in the mix. The real question is for a list, once its parameters are decided upon, is should it be made? With the gamut of personalities here, there's far too many people who wouldn't even stop to ask that question and I've a real concern that crap will flood the forum, to the detriment of the established lists, and we may miss out on some good ideas in that morass.

Hrm...maybe the request for a reg code should include an essay on why that user should be allowed to use NGLister. They'll have to provide conclusive evidence that their list is useful to the NG community.

At 8/27/04 07:54 PM, RedCircle wrote:
At 8/27/04 07:35 PM, ShittyKitty wrote: NOTICE
I've taken NGLister off my website until further notice. I'll be developing a password protection solution to help limit the number of lists on Newgrounds.
Heh...now you make me feel like I have a pirated copy :)

Arr, matey! Did you see my attack on this noob wanna-be pirate?

I was a little afraid of this happening. I mean, it's an incredible breakthrough for these listmakers, but I can see that there would (potentially) be a surge of this happening and the NG bandwidth would suffer for the rest of the site.

I was predicting right after you created this, that it would eventually be one person maintaining ALL the lists on their site.

Yeah, I guess I was letting this new-found popularity get to my head ;-)

Anyway, good job on halting this before it got out of control.

Thanks. I'm working on a locked version as we speak, which will require a registration code to work.

Response to Nglister - Ng Profile Data Miner 2004-08-27 20:20:45


At 8/27/04 07:35 PM, ShittyKitty wrote:
NOTICE
I've taken NGLister off my website until further notice. I'll be developing a password protection solution to help limit the number of lists on Newgrounds.

Don't get me wrong. The Data Miner is a phenomal asset. I just see so much potential for abuse ... it doesn't even have to get into the "wrong" hands, just someone who doesn't think far enough ahead. I should have brought all this up before, I was just too starry-eyed at the concept. Sorry.

At 8/27/04 07:54 PM, D0GMA wrote:
Personally, I don't post or review a lot, so I could see a need for those lists.

Should have read: Personally, I don't post or review a lot, but I could see a need for those lists.

At 8/27/04 07:54 PM, RedCircle wrote:
I was a little afraid of this happening. I mean, it's an incredible breakthrough for these listmakers, but I can see that there would (potentially) be a surge of this happening and the NG bandwidth would suffer for the rest of the site.

Bandwidth/resource usage would only be an issue if that huge input list of profiles SK is generating is made publicly available. Each request for a PID is minimal, but that file will have over 500,000 working PIDs to pull. Get a bunch of twits together with it and they could put a dent in everyone's usage at NG, and it might not even be on purpose. It's like an e-mail virus. Each, by itself, is nothing, but they add up. The most likely issue would be filling the forums with nonsense.

I could see d0gma making several lists on his site. One for XP, VP, so on...with this new NGlister, it wouldn't be too difficult for one person to do and maintain all the lists.

It won't happen. Although gathering the information for any list is a matter of 5-10 minutes, it still took me over an hour to set everything up to be able to post, just as data, those lists. Additionally, there's at least another hour of formatting the pages (inserting table headers, changing links, etc.), plus the time for the write-ups for each list. I maintain 9 lists based upon the same stats, plus the Full List. I really can't see spending the time necessary to format everything to accommodate the addition of VP, XP, and Review lists (or resurrecting Posters), and definitely not the response rate adjunct to the reviewers list.

Response to Nglister - Ng Profile Data Miner 2004-08-27 21:41:00


At 8/27/04 08:20 PM, D0GMA wrote:
At 8/27/04 07:35 PM, ShittyKitty wrote:
NOTICE
I've taken NGLister off my website until further notice. I'll be developing a password protection solution to help limit the number of lists on Newgrounds.
Don't get me wrong. The Data Miner is a phenomal asset. I just see so much potential for abuse ... it doesn't even have to get into the "wrong" hands, just someone who doesn't think far enough ahead. I should have brought all this up before, I was just too starry-eyed at the concept. Sorry.

Don't worry, you weren't the only one who was too caught-up in the action to think of the consequences. Oh, and I didn't get you wrong; I wanted to pull NGLister before anything happened. I've added a registration function to the program; it says that I require an essay, but don't worry; just email me the reg request code it gives you, and I'll email you a reg code.

Oh yeah: you're looking at Newgrounds' newest Elite Guard Sergeant! :-D Now I'll have to get 1,000 b/p's to rank up again...:-( Better get to work!

Response to Nglister - Ng Profile Data Miner 2004-08-27 21:44:54


Whoops, forgot to mention the new NGLister update! Version 1.4 is online, and it's only got one new feature: mandatory registration. I'd really appreciate if people using 1.3 would download the new 1.4; registration is completely free, and if you're an established list-maker, there will be no problem with registration.

If you're using 1.3 and you keep it for yourself or share it with your friends to make extraneous crap lists that nobody cares about.....well, it won't be pretty, that's all I can say. You'd probably either get ignored or banned for posting lists that nobody cares about, anyway. Just do the right thing and get 1.4.

Response to Nglister - Ng Profile Data Miner 2004-08-27 21:46:30


At 8/27/04 09:44 PM, ShittyKitty wrote: Whoops, forgot to mention the new NGLister update! Version 1.4 is online, and it's only got one new feature: mandatory registration. I'd really appreciate if people using 1.3 would download the new 1.4; registration is completely free, and if you're an established list-maker, there will be no problem with registration.

Errrr...forgot to post the URL...I think I'm going too fast tonight... http://www.jfsoftware.com/win/general/nglister.zip. Too much stuff going on at once...

88.1% complete...

Response to Nglister - Ng Profile Data Miner 2004-08-28 00:46:05


sound great :) you saving them so much timei can c y they all saying nice things to you. wish i had one without the code tho :P i just pull them up from chris's list ;)

Response to Nglister - Ng Profile Data Miner 2004-08-28 01:57:08


The 1,000,000 profile miner finished! I made a thread for it here: http://www.newgrounds.com/bbs/topic.php?id=166951.

Response to Nglister - Ng Profile Data Miner 2004-08-28 02:31:37


Official NGLister and NG Total Listing website:

http://www.jfsoftware.com/ng.html

...or just click on my sig pic!

Response to Nglister - Ng Profile Data Miner 2004-08-28 07:24:07


At 8/28/04 02:31 AM, ShittyKitty wrote: Official NGLister and NG Total Listing website:

http://www.jfsoftware.com/ng.html

...or just click on my sig pic!

Looks great Josh!

You may, however want to change your (UPDATED!!!) link in your bbs sig from:

http:///
to
http://

Response to Nglister - Ng Profile Data Miner 2004-08-28 07:34:04


At 8/28/04 07:24 AM, RedCircle wrote:
At 8/28/04 02:31 AM, ShittyKitty wrote: Official NGLister and NG Total Listing website:

http://www.jfsoftware.com/ng.html

...or just click on my sig pic!
Looks great Josh!

You may, however want to change your (UPDATED!!!) link in your bbs sig from:

http:///
to
http://

Oops. Guess I got a little over-excited. It's either that or the lack of sleep.

Response to Nglister - Ng Profile Data Miner 2004-08-28 17:24:10


I wonder if this means I can retire from doing my list :)

I have an old and a new version of the program myself. Havn't had time to run it. Mostly due to the fact I saw the sever suffering.
I am glad you decided to do something before your program became rampant amoung Ng Users.

Response to Nglister - Ng Profile Data Miner 2004-08-28 17:38:47


I know this been address I was accually suprised not to find an X where for example where Shak3s would be on the list fro EXP. AHe would be in the 270-280 range. Obviously since he doesn't have profile your data miner would not find him. I was just curious if you left off the X's people or program missed them.\

Response to Nglister - Ng Profile Data Miner 2004-08-28 20:05:36


At 8/28/04 05:24 PM, ramagi wrote: I am glad you decided to do something before your program became rampant amoung Ng Users.

I'm just sorry I didn't think of doing something earlier. Oh well, at least now I've got a relatively fail-safe method of doing this.

At 8/28/04 05:38 PM, ramagi wrote: I know this been address I was accually suprised not to find an X where for example where Shak3s would be on the list fro EXP. AHe would be in the 270-280 range. Obviously since he doesn't have profile your data miner would not find him. I was just curious if you left off the X's people or program missed them.\

The program probably missed him. It only returns "X" values when a profile exists, but it's missing certain data. Since he has no profile at all, the program skips his profile number completely.

Response to Nglister - Ng Profile Data Miner 2004-08-28 20:11:53


A quick query (haven't really abused the program to see if this can be done already) - would it be possible to have the program export more than one stat into a single text file? Or to have the option of having the input also listed in the output file (such as me being able to see the profile ID as well as the amount of reviews, for example)?

Response to Nglister - Ng Profile Data Miner 2004-08-28 20:31:18


At 8/28/04 08:11 PM, jonthomson wrote: A quick query (haven't really abused the program to see if this can be done already) - would it be possible to have the program export more than one stat into a single text file? Or to have the option of having the input also listed in the output file (such as me being able to see the profile ID as well as the amount of reviews, for example)?

Hrm...the program was mainly designed for Excel spreadsheets, which don't allow copy-and-pasting of multiple columns of text. I suppose I could rig up the program to make a formatted text file (make it look like a table). Perhaps this could be a first actual update with the registration software!

Response to Nglister - Ng Profile Data Miner 2004-08-29 22:08:38


Okay, first... the issue I mentioned briefly in the pentalist topic...

ShittyKitty, I believe that an error in 1.3 (and presumably 1.4 as well) is present that is similar to how you said the Voting Power: x.xx vote(s) error worked... I noticed that both fixit and Syme have 1 review, but the review output file had them both with 0 reviews.

I can only assume that this is due to it saying "1 review" in their profiles instead of "1 reviews" with that s on the end.

If you fixed the vote/votes issue but not the review/reviews issue... I wonder if there's any other areas of the profile where a certain number, or amount of something, could cause the lister to give incorrect output, but I can't think of another one. :::shrugs:::

At 8/27/04 07:35 PM, ShittyKitty wrote:
At 8/27/04 07:23 PM, ShittyKitty wrote: Perhaps I could re-build 1.3 with a password function? A list-builder would request a password from me to run the program within a certain time-frame, and I would generate the password and email it to them. The password would only work during that time-frame. Plus, by only distributing passwords to known list-makers (the ones that worry about too many lists being made), maybe this should help fix the problem. What do you think?
NOTICE
I've taken NGLister off my website until further notice. I'll be developing a password protection solution to help limit the number of lists on Newgrounds.

Um... okay, but I personally think you and Dogma are worrying too much about those issues. There's been nothing to stop people from making "too many lists" in the past, and no admins have ever said a single word about there BEING too many lists, so that isn't a problem. You probably don't know of it, but we used to have a top 25 ppd list around here, for an example of a not-very-important list, and as Dogma says, the pentalist can be considered non-essential since it's not simply an extension of an official NG List (Dogma and jonthomson and ramagi's lists) or the creation of a list of one single NG stat that isn't ranked or listed by NG (saves, blams, VP).

But in the past, very few lists have been made. Take joobie's "3 person 2000+ quadlist" as a jovial example of the occasional non-serious list. I really don't think that readily available access to a program like 1.2 or 1.3 of your NG Lister would cause people who never tried to make a list in the past to suddenly decide they just HAVE to make a list of something and post it on the BBS.

Also, as to the whole "oh, one person can now just do every single list," whoever said that... perhaps you don't realise that, but even WITH this program, there is manual work behind lists. The actual sorting, excel work, and posting, if nothing else... but for most lists, also finding new people to add to the list. Granted, running every single userID# as ShittyKitty has done is a solution to that, and could then be used to create the exp list, the b/p list, the blam and save lists, the VP list, and even my pentalist... BUT... it's incredibly wasteful in terms of time, I would think, to run all those profile #s through every time. And if you DON'T run them all through, how do you ensure you're not missing anyone? Anyway, no system is perfect, but it's ridiculous to say that there's going to be an explosion in new listmakers or a consolidation in current lists under one person. With the exception of jonthomson/ramagi's exp lists morphing into a single list, and me turning over my two original top 50 lists to Dogma (who them expanded them into 8 total lists), there really isn't much of a "list monopoly" issue on Wi/Ht?, I don't think.

Besides, I can't speak for everyone, but MY goal is that one day any list I'd want to create would be done by NG each night. I hope for a top 50 (at least) VP list, and more importantly, VP rankings in everyone's profile... someday. Then I'd stop making the list, it's just that simple. We make lists to fill gaps that NG itself doesn't fill. We don't make them because we like to feel important or bullshit like that. I'd love to no longer be doing the VP list, and see a fresh top 50 VP every morning when I wake up, and automatically know my VP rank every day due to looking at my profile.

At 8/28/04 08:11 PM, jonthomson wrote: A quick query (haven't really abused the program to see if this can be done already) - would it be possible to have the program export more than one stat into a single text file? Or to have the option of having the input also listed in the output file (such as me being able to see the profile ID as well as the amount of reviews, for example)?

What's the need for that? Just paste the profile ID list from your input file into column A of a fresh .xls and then paste the output next to it in column B, and bam, you've got instant line-up and you can see the cause and effect next to each other. Add another column with the name output if you want to see who's connected to each id and stat, of course.

OR... do what I do when I'm making a quick spot-check. I open both the id (or name) file and the stat output file next to each other, both in notepad, and then the lines line up fine as long as I position the windows correctly. If I need to see something not near the top of the files, just pagedown the same number of times in both notepad windows and there ya go.


gfox // wi/ht?#9 // defunct PentaList (final update: 3/15/2008) // Cyberdevil's HexaList!

a long, long time ago: 60000 b/p (#2) // 36000 blams (#3) // 24000 saves (#1)

BBS Signature

Response to Nglister - Ng Profile Data Miner 2004-08-29 22:15:54


At 8/29/04 10:08 PM, gfoxcook wrote: (who them expanded them into 8 total lists)

argh... who THEN expanded them, obviously.

I meant to add, BTW, but got distracted by trying to squeeze in my reply to jon's post as well... that because of everything I said about there being no real list-monopoly issue, and that if people are going to make unnecessary lists now with an unsecured version of NG Lister, those same people would have made them in the past by manually pulling profiles like me, Dogma, ramagi, jonthomson, NGling, etc. etc. all have done (the fact that so many people have spent a lot of time and effort doing the lists the MANUAL way should prove my point that this program doesn't suddenly enable everyone to make lists... there's nothing special about the people who make lists, manually or otherwise, they just like a stat and see a useful/informative service they can provide to other users... and most of us started making the lists for ourselves, as well)...

Anyway, that because of all that, and the lack of admin condemnation of silly lists, that the ONLY issue remaining that's a valid issue to discuss with regard to the NG Lister software is INDEED malicious potential upon the NG database server. Dozens of people running the program simultaneously, and checking more than 1000 IDs, say, would indeed probably have deleterious effects on NG as a while in some way.

And so, I can understand the changes you've made in 1.4 from THAT angle, but not from the other ones that have been discussed on page 3 of this topic, that's all.


gfox // wi/ht?#9 // defunct PentaList (final update: 3/15/2008) // Cyberdevil's HexaList!

a long, long time ago: 60000 b/p (#2) // 36000 blams (#3) // 24000 saves (#1)

BBS Signature

Response to Nglister - Ng Profile Data Miner 2004-08-29 22:55:58


At 8/29/04 10:08 PM, gfoxcook wrote: Okay, first... the issue I mentioned briefly in the pentalist topic...

ShittyKitty, I believe that an error in 1.3 (and presumably 1.4 as well) is present that is similar to how you said the Voting Power: x.xx vote(s) error worked... I noticed that both fixit and Syme have 1 review, but the review output file had them both with 0 reviews.

I can only assume that this is due to it saying "1 review" in their profiles instead of "1 reviews" with that s on the end.

If you fixed the vote/votes issue but not the review/reviews issue... I wonder if there's any other areas of the profile where a certain number, or amount of something, could cause the lister to give incorrect output, but I can't think of another one. :::shrugs:::

Err...bloody buggy code. Let's see (I've replaced the greater-than and less-than signs on the HTML tags with parentheses):

* Name:
---has (title)Personal Profile: before it, and (/title) after it
---I haven't found an exception yet...
* Experience Level:
---has (a href="javascript:OpenGoldExplainWin();")LV before it, and (/a)(/font) after it
---Again, no exceptions yet.
* Experience Points:
---has Exp. Points: before it, and a space after it
---I've checked this using ceagle's profile, and it's the same with 0 points.
* Reviews:
---has a greater-than sign before it, and review after it
---This is where the problem was: the program was looking for reviews available after it.
* Submissions:
---has (SELECT NAME="id" onChange="JumpToMovie();) before the list, has (OPTION before each entry, and has (/SELECT) at the end of the entries
---If the program can't find (SELECT NAME..., then it assumes that submissions = 0.
* Audio:
---Same as submissions, but JumpToAudio(); instead of JumpToMovie();
* Stickers:
---has (b)Stickers:(/b) before it, and ( after it (an actual parenthesis)
---Most profiles have 0 sticker orders, so this always works.
* Voting Power:
---has (b)Voting Power:(/b) before it, and vote after it
---Again, like with the Reviews, the program used to look for votes after it.
* Blams:
---has Involved in the termination of before it, and crappy entries. after it
---Even 0 blam profiles have a plural "entries" in them.
* Blams:
---has Assisted in the protection of before it, and quality entries. after it
---Even 0 save profiles have a plural "entries" in them.
* BBS Posts:
---has BBS Posts:(/a)(/font) before it, and in total after it
---Even 0 post profiles have a plural "Posts" in them
* Batting Avg:
---has (b)Batting average(/b): before it, and / 5.00 ( after it (an actual parenthesis)
---The program knows that some profiles have no batting average, so it doesn't dork out over these; those profiles with averages will have the / 5.00 in them.
* Exp Rank:
---has Experience:(/b) Ranked # before it, and out of after it
---Already fixed the absent experience rank problem; otherwise, all other profiles have this text in them.
* B/P Rank:
---has Blam/Protect:(/b) Ranked # before it, and out of after it
---Already fixed the absent blam/protect rank problem; otherwise, all other profiles have this text in them.
* Sign-Up Date:
---has Sign-up Date: before it, and a tab-movement after it
---At first, the program was looking for a space after it; now it looks for a tab, and everything works.

If you find anything different from what I've posted above, please inform me. Other than that, I've released NGLister 1.4.1 (with the review count fix) at its usual place: http://www.jfsoftware.com/ng.html. Thanks for the feedback!

Um... okay, but I personally think you and Dogma are worrying too much about those issues. There's been nothing to stop people from making "too many lists" in the past, and no admins have ever said a single word about there BEING too many lists, so that isn't a problem.

Unfortunately, nothing like this program has been done before. I just don't want to be responsible for the mess that this could possibly create.

But in the past, very few lists have been made. Take joobie's "3 person 2000+ quadlist" as a jovial example of the occasional non-serious list. I really don't think that readily available access to a program like 1.2 or 1.3 of your NG Lister would cause people who never tried to make a list in the past to suddenly decide they just HAVE to make a list of something and post it on the BBS.

Well, if you think about it, why do some people decide that they just HAVE to make some crappy stick flash and post it to the Portal? They know there's nothing good about the movie they've made, they just want to post it for kicks, and to make everyone mad. If people like this got their hands on an easy list maker, they could pull a profile list out of another list and post their own. Or they could put their own profile in there tons of times and kill Newgrounds' bandwidth by pulling their own profile umpteen million times.

Although, now that I have thought about it, the NGLister wouldn't be terribly easy to start a list with. Eh, better safe than sorry.

Response to Nglister - Ng Profile Data Miner 2004-08-29 23:21:27


At 8/29/04 10:55 PM, ShittyKitty wrote:
At 8/29/04 10:08 PM, gfoxcook wrote: I can only assume that this is due to it saying "1 review" in their profiles instead of "1 reviews" with that s on the end.

(snip)

* Reviews:
---has a greater-than sign before it, and review after it
---This is where the problem was: the program was looking for reviews available after it.

Figured as much.

(snip)

If you find anything different from what I've posted above, please inform me. Other than that, I've released NGLister 1.4.1 (with the review count fix) at its usual place: http://www.jfsoftware.com/ng.html. Thanks for the feedback!

I take it you didn't find any other instances of error potential in that huge block of stuff? I scanned most of it, but since there were no spaces at all, it was kinda hard to process. O_o

Anyway, I'll DL 1.4.1 in a bit, but how do I go about operating it? If I need a code or whatever, can you please e-mail it to me (gfoxcook@newgrounds.com)?

Unfortunately, nothing like this program has been done before. I just don't want to be responsible for the mess that this could possibly create.

I was just trying to emphasize that, to me, the ONLY mess would be liljim getting pissed off that the database server was getting knocked around way too much. You COULD just e-mail him at liljim@newgrounds.com and find out what he thinks about your mining of all 960,000+ profile ID#s, yanno. O_o

Well, if you think about it, why do some people decide that they just HAVE to make some crappy stick flash and post it to the Portal? They know there's nothing good about the movie they've made, they just want to post it for kicks, and to make everyone mad.

Those people are attention whores. Attention whores, in the past, have been much more likely to create shitty movies and upload them to the portal, or to spam up General with stupid topics. Attention whores don't tend to go to the trouble of making extensive stat lists, with or without your program.

If people like this got their hands on an easy list maker, they could pull a profile list out of another list and post their own.

And the thread would then be deleted or locked by myself or another mod. No problem there.

Or they could put their own profile in there tons of times and kill Newgrounds' bandwidth by pulling their own profile umpteen million times.

Like I said, that is the ONLY issue. And you could find out the potential effects by talking to liljim about it.

Although, now that I have thought about it, the NGLister wouldn't be terribly easy to start a list with. Eh, better safe than sorry.

Exactly. It doesn't "make a list for you," it works with preexisting excel-based lists like the ones it's already been used with. Someone would have to make a huge input file (like you did, I presume) to even begin to create havoc with NG's servers. And do a lot more than just that to make an actual list for the BBS. And again, moderators would take care of any insanity posted to the BBS. The database is the ONLY issue, again. Talk to liljim. Again. That is all. #;-}>

Oh, and send me the code or whatever, pretty please, so I can try out the review fix. #;-}>


gfox // wi/ht?#9 // defunct PentaList (final update: 3/15/2008) // Cyberdevil's HexaList!

a long, long time ago: 60000 b/p (#2) // 36000 blams (#3) // 24000 saves (#1)

BBS Signature

Response to Nglister - Ng Profile Data Miner 2004-08-29 23:30:15


At 8/29/04 11:21 PM, gfoxcook wrote: I take it you didn't find any other instances of error potential in that huge block of stuff? I scanned most of it, but since there were no spaces at all, it was kinda hard to process. O_o

Sorry, I was trying to get it out as fast as I could (it still took 40 minutes), so I wasn't too worried about formatting. But no, I don't think I've found anything wrong. I'm a little worried about the sign-up date (I haven't tested it too much), but the other stuff feels okay. Of course, I haven't really put the whole thing through its paces yet (you know, running 15 different stat lists for all 960,000 profiles). That would be a true nightmare. :-O

Anyway, I'll DL 1.4.1 in a bit, but how do I go about operating it? If I need a code or whatever, can you please e-mail it to me (gfoxcook@newgrounds.com)?

Download 1.4.1 and run the NGLRegRequest program in the ZIP file. It'll give you a reg request code, which you can email to me at joshua_70448 (at) ureach.com, and I'll email you back your reg code. I'll need your request code to make your reg code, first.

I was just trying to emphasize that, to me, the ONLY mess would be liljim getting pissed off that the database server was getting knocked around way too much. You COULD just e-mail him at liljim@newgrounds.com and find out what he thinks about your mining of all 960,000+ profile ID#s, yanno. O_o

Shh! Maybe if I don't mention it, he won't notice! 'Sides, I only run through the whole list once a month; the other updates are only 14,119 (b/p) and 95,765 (exp) profiles.

Like I said, that is the ONLY issue. And you could find out the potential effects by talking to liljim about it.

Yeah, I spent 40 minutes copying and pasting stuff to make my message, so I missed yours. Oops.

Oh, and send me the code or whatever, pretty please, so I can try out the review fix. #;-}>

Just send me that code!

Response to Nglister - Ng Profile Data Miner 2004-08-30 00:22:45


At 8/29/04 11:30 PM, ShittyKitty wrote: Sorry, I was trying to get it out as fast as I could (it still took 40 minutes), so I wasn't too worried about formatting. But no, I don't think I've found anything wrong. I'm a little worried about the sign-up date (I haven't tested it too much), but the other stuff feels okay. Of course, I haven't really put the whole thing through its paces yet (you know, running 15 different stat lists for all 960,000 profiles). That would be a true nightmare. :-O

I haven't tested sign-up date yet either, mostly because I'm not going to add it to either of my excel files, so... there's not much point. However, maybe I'll give it a spin as just a test and see what happens.

If you're going to run all 15 stats, you might as well run them on a select group. All 960k+ would be totally insane indeed. o_o

Download 1.4.1 and run the NGLRegRequest program in the ZIP file. It'll give you a reg request code, which you can email to me at joshua_70448 (at) ureach.com, and I'll email you back your reg code. I'll need your request code to make your reg code, first.

Yeah, I figured all of that out about 5 minutes after making the above post. Silly me.

Shh! Maybe if I don't mention it, he won't notice! 'Sides, I only run through the whole list once a month; the other updates are only 14,119 (b/p) and 95,765 (exp) profiles.

He probably already knows about it. He DOES read the BBS, you know. HE KNOWS ALL.

Oh, and send me the code or whatever, pretty please, so I can try out the review fix. #;-}>
Just send me that code!

I did! Almost an hour ago. Have you gotten it? If not, perhaps I should just say the code here. You said it's machine specific, right? So it shouldn't really matter.


gfox // wi/ht?#9 // defunct PentaList (final update: 3/15/2008) // Cyberdevil's HexaList!

a long, long time ago: 60000 b/p (#2) // 36000 blams (#3) // 24000 saves (#1)

BBS Signature

Response to Nglister - Ng Profile Data Miner 2004-08-30 00:32:32


At 8/30/04 12:22 AM, gfoxcook wrote: I haven't tested sign-up date yet either, mostly because I'm not going to add it to either of my excel files, so... there's not much point. However, maybe I'll give it a spin as just a test and see what happens.

If you're going to run all 15 stats, you might as well run them on a select group. All 960k+ would be totally insane indeed. o_o

Nah, that's what you guys are for. Beta testers! ;-)

I did! Almost an hour ago. Have you gotten it? If not, perhaps I should just say the code here. You said it's machine specific, right? So it shouldn't really matter.

Sorry, I've been busy fighting with webpage scripting. Stupid JavaScript... I was updating my NGLister update, and I got a little caught-up in it.

Not too good at following directions, though? I noticed you followed the registration program's instructions (you had an "essay"), but you didn't send your NG username!! Lucky for you, your email username is the same as your NG one. Oh well, since you don't like email too much, I'll post your reg code here. Don't worry, nobody else can use it; it's made for you and you only: 04E9916783

Response to Nglister - Ng Profile Data Miner 2004-08-30 00:45:14


At 8/30/04 12:32 AM, ShittyKitty wrote: Nah, that's what you guys are for. Beta testers! ;-)

Ayup.

Sorry, I've been busy fighting with webpage scripting. Stupid JavaScript... I was updating my NGLister update, and I got a little caught-up in it.

The link you just provided leads to a short snippet of (visible) HTML code. O_o

Not too good at following directions, though? I noticed you followed the registration program's instructions (you had an "essay"), but you didn't send your NG username!!

I scanned the stuff, but the part I took as "instructions on what to do" was this:

"To submit your registration request, email the above registration request code and your essay to joshua_70448@ureach.com.""

No mention of username there, yanno. Just request code and essay. ;_;

Anywho, looking at the larger paragraph above that last line, I see it now. Whoops.

Lucky for you, your email username is the same as your NG one.

Not lucky. By design. I've had dozens of usernames on dozens of sites and ISPs and chat programs and so forths and so ons since I got online in 1994, and all of them have been... gfoxcook.

Also, why would gfoxcook on NG not have gfoxcook@NG as his NG e-mail? Then again, perhaps some other mods don't have their actual username as e-mail (but that's usually because they change their name all the time).

Oh well, since you don't like email too much, I'll post your reg code here. Don't worry, nobody else can use it; it's made for you and you only: 04E9916783

I never said I didn't like e-mail. I said I was feeling too lazy to use it tonight. #;-}> Anywho, thanks for getting that to me whichever way.

One thing regarding your site: you say that NG ranks ties at the low end of the rank... i.e. 1, 3, 3, 4 for two people in the middle with the same value...

but that's not really true. I've been tied in exp with plenty of people, both recently and long ago, and we had expranks based on our signup dates (or if you prefer, userID#s). For instance, almost every single day, I'm tied with Miikro in exp (some days he/whoever's depositing for him doesn't deposit until after the profile-rank-update-time, but most days he does). And for the past week, I've been ranked #179 and he's been ranked #180. Were you referring to b/p? Cause it's obviously not true that ties share the same rank in exp. I don't think I've ever been tied with anyone in b/p at the exact moment of profile-rank-update-time, so I wouldn't know on that.


gfox // wi/ht?#9 // defunct PentaList (final update: 3/15/2008) // Cyberdevil's HexaList!

a long, long time ago: 60000 b/p (#2) // 36000 blams (#3) // 24000 saves (#1)

BBS Signature

Response to Nglister - Ng Profile Data Miner 2004-08-30 00:58:35


At 8/30/04 12:45 AM, gfoxcook wrote: The link you just provided leads to a short snippet of (visible) HTML code. O_o

Umm...really? Dang...that's my attempt at shortening the links in my BBS sig so I can fit more stuff in it. It's been working for me, but I guess I'll have to add a little more to it...

Lucky for you, your email username is the same as your NG one.
Not lucky. By design. I've had dozens of usernames on dozens of sites and ISPs and chat programs and so forths and so ons since I got online in 1994, and all of them have been... gfoxcook.

Yeah, same here. I usually use joshua_70448 (or derivatives thereof) for most sites, but I decided to get a little creative here. Plus, most places don't allow dirty words in your username ;-)

One thing regarding your site: you say that NG ranks ties at the low end of the rank... i.e. 1, 3, 3, 4 for two people in the middle with the same value...

but that's not really true. I've been tied in exp with plenty of people, both recently and long ago, and we had expranks based on our signup dates (or if you prefer, userID#s). For instance, almost every single day, I'm tied with Miikro in exp (some days he/whoever's depositing for him doesn't deposit until after the profile-rank-update-time, but most days he does). And for the past week, I've been ranked #179 and he's been ranked #180. Were you referring to b/p? Cause it's obviously not true that ties share the same rank in exp. I don't think I've ever been tied with anyone in b/p at the exact moment of profile-rank-update-time, so I wouldn't know on that.

Hrm...maybe I was wrong. I just based that off of my b/p rank between Total Listing and NG. Blah. I like my way, anyway; it's consistent, and it makes sense.

Response to Nglister - Ng Profile Data Miner 2004-08-30 01:03:29


At 8/30/04 12:58 AM, ShittyKitty wrote: Yeah, same here. I usually use joshua_70448 (or derivatives thereof) for most sites, but I decided to get a little creative here. Plus, most places don't allow dirty words in your username ;-)

Why the 70448?

Hrm...maybe I was wrong. I just based that off of my b/p rank between Total Listing and NG. Blah. I like my way, anyway; it's consistent, and it makes sense.

Like I said, maybe you were talking about b/p rank... you didn't specify that on the page, though...

and what I'm trying to tell you is that your way is NG's way, with regard to experience rank, at least. I don't know why b/p rank isn't working the same way. I've never personally seen two people have the same b/p rank in their profiles on the same day due to being tied at the time profile ranks updated, but... I'll take your word on that. Maybe the ranking/sorting algorithmthingie doesn't want to bother with breaking the tie in b/p for some reason. But I can once again assure you that experience ties ARE broken and no two people can have the same exp rank on the same day. BE ASSURED!

Oh, one more question: does the separate .exe used for the code thingie in 1.4+ need to be kept after you've entered the code? Or can you delete it? Or what? Just curious.


gfox // wi/ht?#9 // defunct PentaList (final update: 3/15/2008) // Cyberdevil's HexaList!

a long, long time ago: 60000 b/p (#2) // 36000 blams (#3) // 24000 saves (#1)

BBS Signature

Response to Nglister - Ng Profile Data Miner 2004-08-30 01:07:53


At 8/30/04 01:03 AM, gfoxcook wrote: Why the 70448?

It was my old ZIP code. When I first signed up for Yahoo! years ago, it suggested that.

Oh, one more question: does the separate .exe used for the code thingie in 1.4+ need to be kept after you've entered the code? Or can you delete it? Or what? Just curious.

Trash it! It's useless, unless you dig around in the registry and delete where the reg code is saved. Or if your registry gets wiped out. But if you registry dies, I think you'd have bigger problems than NGLister not working. ;-) The only reason I made it separate was because I didn't feel like building it into the main program...I'm lazy! :-O