At 3/21/05 02:46 AM, SeeD419 wrote: You're prize: Looking like an idiot on the NG Forums!

"Your".

Seriously though, I guess you must have overlooked the word "ALMOST".

I didn't. Being 104 days off isn't ALMOST a year dumbass.

I wanted to know when I last missed a deposit, what I didn't want was some random user who for sure can't tell me what I'm looking for giving me an already completely obvious answer...

Wade is trying to debate with us about the state of stat threads, what he doesn't want is for some random user looking for the answer to a completely pointless question.

Oh boy, do I feel stupid. :|

You fail.

Now that everyone (for the most part) seems to have finally cooled down I have a small handful of things to say.

First, technical stuff regarding the database errors. I don't know what a code 3 error necessarily signifies, but I'm under the impression that any of the errors we are getting can be remedied (at least partially, if not wholly) by tweaking with MySQL settings. My first guess regarding the database errors is that there are too many connections: according to this page, "The maximum number of connections MySQL can support depends on the quality of the thread library on a given platform. Linux or Solaris should be able to support 500-1000 simultaneous connections, depending on how much RAM you have and what your clients are doing. Static Linux binaries provided by MySQL AB can support up to 4000 connections." Is Newgrounds capable pulling over 1000 unique requests at an instant in time? I'd believe it. More than 4000? I'm not so sure about that, at a single instant. Nonetheless, I wonder if the max_connections option is at its optimum setting. Of course, this will be related to the table_cache setting as well.

Next, I don't see why we couldn't come to a compromise regarding SK's list. What I have in mind is basically a text dump of the public profiles -- a newline-delimited file containing space-delimited userid, alias, sign-up date, experience, blams, saves, and review count. For a worst-case scenario, assume the userid is 7 digits, the alias is 20 characters, the sign-up date is 8 characters in the form of MMDDYY, and the rest of those numbers are 5 digits at most. All this plus spaces and a newline character comes out to 60 characters of data for each user. The next worst case is that every one of the 704,757 users has a public profile, so 704757 * 60 = 42,285,420 bytes of plaintext, mostly numbers, that will compress excellently using any compression utility.

I think with all the data I listed, SK can do everything that he was doing before. It might also be a good idea to put a line in with the date of the file's generation, and the next scheduled time of generation, because we certainly don't want people pulling ~40MB (worst case, uncompressed) of data pointlessly; moreover, it would be trivial to use a PHP script (include the plaintext file which is stored outside of the HTTP docroot) that only allowed downloads in off-peak hours; use an obscure URL that James will email to "pre-approved" listmakers, and I think the problems involving lists are solved. If SK can write something (in VB, since you distributed it, isn't it?) to parse individual HTML pages, he can write something to parse a delimited file as written above in one sitting. And on James' end, I doubt it would take him more than an hour to write the above-described script and set up the cronjob to periodically update it.

We've gotten this far without these stats being hosted "officially" on Newgrounds, and this "temporary" solution has been more than adequate for us IMO. If we could continue to have our stats in this fashion, I don't think any of us stat-whores would mind if it takes several years for there to be an official Newgrounds page for any of these stats. Then, James can still concentrate on what will affect more users, and none of the admins have to worry about database errors caused by scripts (by the way, I started after the first instances of the errors showed up, and I ended promptly when Wade said to stop. I can easily continue the updates by hand now, although with this thread as it is, I don't seem to have much of a place to post anything. All I'm saying, though, is that the database errors are not solely caused by scripts.). Tangent aside, I believe my solution is win-win for both parties, but feel free to point out flaws as you see them.

This last part will be directed more towards James again. I completely understand the benefits that you have in mind for working alone on a large project such as Newgrounds. But when you say that you spend the most time making repairs and tweaks to existing scripts, that leads me to believe that it's largely busywork (testing, rewriting, testing, etc.) that can be somewhat avoided. I assume that in your testing, you identify bottlenecks. Is there a reason you don't use the resources in the programming forum? There's a bunch of bright minds in there who would be more than happy to help out, even uncredited.

I'm thinking along the lines of you coming in and saying "Here's this part of my script; it needs to take in this array of arrays of form (string, int, int) and return 3 arrays where each one has the original data set sorted on each value. My script runs through a set of 2000 (random data) arrays in about 200ms on average, but I'd really like to get that down under 80-100ms." As long as it wasn't "needs to be done yesterday" urgent, you could probably get most of those kinds of tweaks done in a reasonable amount of time by us. We are happy to help out (for free, no less), and you spend only a little time double-checking the quickest function rather than spending more time making it yourself. Another win-win IMO, but I have a feeling you'll get me on some details on this one.

Oh, and since we were on the topic of spelling errors, I noticed that the text in the agreement box when posting is missing a vital "e" in the word "infringments" [sic].

At 3/21/05 04:11 AM, pieoncar wrote: Now that everyone (for the most part) seems to have finally cooled down I have a small handful of things to say.

Christ, by the looks of your profile I figured you were just another j00bie, but I must say, you're pretty damn knowledgeable about the subject. I'm glad you're finding ways to continue SK's lists.

At 3/21/05 04:07 AM, Seizure_Dog wrote: "Your".

Oh for fuck's sake, quit it. I already have an english teacher.

I didn't. Being 104 days off isn't ALMOST a year dumbass.

Over 2/3 is close enough for me, dumbass.

Wade is trying to debate with us about the state of stat threads, what he doesn't want is for some random user looking for the answer to a completely pointless question.

Random user? Fuck you.

Oh boy, do I feel stupid. :|

You fail.

Faggot.

At 3/21/05 04:32 AM, SeeD419 wrote:

Sometimes a smile says everything you want to say.

:)

At 3/21/05 04:11 AM, pieoncar wrote: I'm under the impression that any of the errors we are getting can be remedied (at least partially, if not wholly) by tweaking with MySQL settings.

They're already extensively tweaked. The database errors that lasted 10 minutes earlier yesterday that gfoxcook has mentioned in this thread were due to me taking the database down so that it incorporated yet further tweaks (any change to the configuration file takes a reboot of the software so that those changes take place - and taking down the db takes time when there are so many concurrent users). Any sustained downtime like that is usually due to me taking the database offline.

Oh, and since we were on the topic of spelling errors, I noticed that the text in the agreement box when posting is missing a vital "e" in the word "infringments" [sic].

Thanks, I've fixed that.

I have noticed that the topic search bar doesn't pick up certain words, the one I found was the word another. Im not sure if this why this is. Is it just left over from when the search functions were tweaked?

At 3/21/05 07:41 AM, Seizure_Dog wrote:

At 3/21/05 04:32 AM, SeeD419 wrote:

Sometimes a smile says everything you want to say.

)

Whatever dude...I'm not continuing this nonsense...

The argument is twice as pointless as the original question.

Therefore, I'm giving you the same thing. Enjoy:

:)

At 3/21/05 02:36 PM, SeeD419 wrote:

At 3/21/05 07:41 AM, Seizure_Dog wrote:

At 3/21/05 04:32 AM, SeeD419 wrote:

Sometimes a smile says everything you want to say.

)

Whatever dude...I'm not continuing this nonsense...

The argument is twice as pointless as the original question.

Therefore, I'm giving you the same thing. Enjoy:

)

Fuck. Doing the same thing here, as you did in the programming forum. and liljim if you ever want to hire a fellow PHPer go for Pilot-Doofy or......me......(no). lol.

At 3/21/05 03:33 PM, Love_Daddy wrote: Fuck. Doing the same thing here, as you did in the programming forum. and liljim if you ever want to hire a fellow PHPer go for Pilot-Doofy or......me......(no). lol.

What are you referring to? I haven't visited the programming forum in months. If you're talking about fighting, 99.9% of my posts are friendly conversation.

Secondly, who are you? Have we spoken before or were you just going through my posts?

At 3/22/05 12:19 AM, SeeD419 wrote:

What are you referring to? I haven't visited the programming forum in months. If you're talking about fighting, 99.9% of my posts are friendly conversation.

Secondly, who are you? Have we spoken before or were you just going through my posts?

No, i just remember how you bugged everyone in the programming forum's regular lounge.

99% in the General but not in our forurm.

At 3/21/05 12:40 AM, WadeFulp wrote: As far as hiring help for James. We have offered to hire someone to help him, but he explained to us it wouldn't really help, and I totally agree with him. To many cooks in the kitchen spoil the soup. If we have some other guy coding things for the site, those are scripts and things that Liljim doesn't know about, or would have to review. So he'd basically spend time watching someone else work. In order for Liljim to keep this site running smoothly, he has to know the entire system like the back of his hand. If someone else is doing things, it would make it very hard for him to know why certain things are happening, he'd have to investigate, etc.

IOW, they'd be stepping all over each other's feet all the time, eh?

I see what you mean. But it seems like there are certain areas of the site where someone could help liljim out by doing all the time-wasting gruntwork, and then show the work to liljim for a quick approval before it hits the site. But if that doesn't jive with liljim's adminning/programming style, I completely understand as well.

liljim is a wizard and if something spoils the magic, it's better not even being attempted.

It's kinda like if you were buliding your own hot rod, then one day someone else was working on it when you weren't there and you go in the following day and are like "WTF?! What's this? How did this get there? What does this do? UHG!" and spend all day figuring out what the hell changed. :)

Or if you have your home all perfectly arranged the way you want it... and then your girlfriend hires a maid to come in and clean shit up and the maid moves all your shit into the wrong drawers and closets and so forth. #;-}>

...

... WARNING: "APPARENT END OF FLAMES BETWEEN ME AND RECON" SECTION OF MY REPLY, AVOID READING IF YOU DON'T CARE, as over 99% of you undoubtedly feel re: this.

At 3/21/05 12:49 AM, ReconRebel wrote:

At 3/21/05 12:37 AM, gfoxclock wrote: It seems you tired of kissing Wade's ass and decided to move on to liljim's. He doesn't really like that, BTW.

This is real mature. How old are you? Just for the record, I've responded to less than 5 posts by Wade and/or liljim in almost 3 years. I'd hardly call that ass kissing. The man just got back from Japan and still took the time to respond in this thread. He mentioned he was tired in his "Noodles" thread (which I didn't post in) so I threw something into my post here. I guess those lists of yours were like a legacy 'eh? My god gfox, how will you and your OCD ever survive now? Get the last word in. Say whatever you want, I won't respond. I've had it with you and your anal attitude.

Of course it wasn't mature, Recon. I was making a petty little spiteful joke to counter all the petty little spiteful shit you seemed to be slinging at me (you know, shit about my "whining" and my longwindedness and how horrible it makes you feel to BE FORCED AT GUNPOINT to read my horribly-written posts).

And I'm 27, as you well know if you've seen my profile in the past 8 months. So what? You're older than me, not younger. Seeing as how you were acting immaturely FIRST for the past week about stats on this forum, I don't think age is a good thing for you to bring up as an excuse/weapon/whatever the hell you were trying to do right there.

I'll admit that "an eye for an eye" isn't the grown-up, mature, responsible thing to do, but I was tired of hearing REPULSIVE shit about myself and about Dogma from you... UNDESERVED INSULTS, in my eyes... and not lowering myself to that level. Tired of turning the other cheek, so to speak.

So I went ahead and lowered myself to that level.

I find it strange that that's the only part of my post you replied to, BTW. But whatever, I'm fine arguing about this for 2 more weeks or simply not at all. I really don't care either way, so... yeah. [EDN]

... END WARNING

...

At 3/21/05 04:11 AM, pieoncar wrote: We've gotten this far without these stats being hosted "officially" on Newgrounds, and this "temporary" solution has been more than adequate for us IMO. If we could continue to have our stats in this fashion, I don't think any of us stat-whores would mind if it takes several years for there to be an official Newgrounds page for any of these stats.

That was a conversation back in January. A lot of people seemed to wish the user-run lists era could continue forever. A lot of people wanted the new NG-uber-list era to begin.

Personally, I said, back then... 2 months ago... that I was split. Of course I'm going to miss the good ol' days...

BUT... if NG provides similar (or even more/better) lists and statistics and rankings than Dogma, myself, jonthomson, ramagi, RC, MW, and SK were providing... then hell no, I'm not going to complain.

And thus, I must disagree with you about the "it would be fine if we could go back to the old way and wait several years for official NG stats pages."

I've already shut it down. I've done lists on NG for over 2 years now. It's taken a lot of time and effort. And it was well worth it. But I'd rather spend my time elsewhere. I already made a decision like that when I turned over my top 50 blams and top 50 saves lists to Dogma in July 2004. Cutting out the VP list this month... and changing the pentalist from 180 people to a much more elite and restricted 30 people... is just the final (presumably) step in my "shutting down the timewasting list-updating cause."

I'm down to one list. One simple, valuable, and popular list (popular even amongst people who dislike all the other lists I've ever done, in fact, like poxpower). That doesn't require too much time to update any longer.

So I don't personally care anymore about the whole NG lister issue at all.

I'll just carry on with my one single, manually-made list, and wait for news from liljim about the arrival of a new statly era in NG history. I look forward to that day, but I won't say "I can't wait." Because I can.

At 3/22/05 03:50 PM, Love_Daddy wrote: No, i just remember how you bugged everyone in the programming forum's regular lounge.

Yeah, like I said, those were my first posts ever. I decided for some reason or another to be an asshole.

Apologeez tew yoo

99% in the General but not in our forurm.

? Most of my posts are in Wi/Ht, I believe. If not then C&C.

At 3/22/05 10:58 PM, SeeD419 wrote:

Apologeez tew yoo

Aight were cool now but i know some of the other people can hold a pretty bad grudge.

At 3/15/05 11:02 AM, liljim wrote: People generally aren't interested in this stuff, there are just a select few who are interested in it.

Well the argument could be made that the word just isn't getting out. I didn't find these stat sites until well into my eighth month on NG (I was directed to them.) and I checked this site thoroughly when I joined. I'm sure there would be more hits if it were locally advertised. Wi/Ht? really doesn't count as site wide advertisement.

At 3/21/05 12:40 AM, WadeFulp wrote: As far as hiring help for James. We have offered to hire someone to help him, but he explained to us it wouldn't really help, and I totally agree with him. To many cooks in the kitchen spoil the soup.

So maybe we should invest ways to make a "super-cook". Bah, bad joke. What we really need to be looking at, not anytime soon hopefully, is that future employees of NG. If someone comes in to do new programming it would be helpful if they had a head start on what James has been doing and his style. Eh, it's a topic for another day.

I found these sites by doing a google search on 'deslona' just to see what would show up. And I found myself on one of the sites (think it was D0GMA's first)

Not im very interested in the results etc... But I found this out only about a month ago.
I hope the whole thing will be redone on NG, or some other site. It was really nice to know stuff.

You hit the nail on the head, that is how it's done. You could have just emailed me, you know.

The offer still stands if anyone wants a list of their topics or any particular stats. If it's topics you want, I'll email them, because the common reaction seems to be "I was such a n00b when I started." Nobody else needs to know how much of a n00b you were (and for the most part, we all were).

At 3/21/05 04:11 AM, pieoncar wrote: Next, I don't see why we couldn't come to a compromise regarding SK's list. What I have in mind is basically a text dump of the public profiles -- a newline-delimited file containing space-delimited userid, alias, sign-up date, experience, blams, saves, and review count. For a worst-case scenario, assume the userid is 7 digits, the alias is 20 characters, the sign-up date is 8 characters in the form of MMDDYY, and the rest of those numbers are 5 digits at most. All this plus spaces and a newline character comes out to 60 characters of data for each user. The next worst case is that every one of the 704,757 users has a public profile, so 704757 * 60 = 42,285,420 bytes of plaintext, mostly numbers, that will compress excellently using any compression utility.

The thing is, both SK and I (and I assume most of the Programming Forum bods) could write programs that would interpret such data easily into custom lists like the ones that have currently been ceased - and since the data would be processed on Local Machine, rather than making numerous connections to NG, there would be no need for database strain. The file(s) would only probably be downloaded by 5 - 10 people, so bandwidth wouldn't be a major issue; and the file(s) would only need to be updated once a week (although once a day would be preferable for lists such as D0GMA's minis) - again, processing even the full list on Local Machine would take a fraction of the time that the programs formerly took.

I do like this idea - I'm glad that NG is going to expand the official lists, but there are still people who will want to create their own interpretations, and your suggestion would solve problems for NG and the listmakers - assuming it wouldn't require too much work form James to output such files.

I just programed a full set of tools that manipulate and display all of your beloved stats. Its called NG tools and its still in BETA and others on the programming forum are testing it now. I will release it in the general forum very soon but if you can't wait you can download it from the programming forum now(not recommended because of bugs). This program Displays all of your stats regarding B/P points,exp, lv and days(months, weeks, or years) until your next lv up for all of your levels. It was inspired by Nemisises program but my is a lot more advanced and has more features. This program Does not use any thrid party scripts so its safe and doesn't bother NG. Oh if you have programming exepraince and feel like beta testing the bastard it can be found here!

Sorry to bump this topic, but I'd just like to say that maybe if NG was using a more reliable database system than mySQL, it would probably be much faster and much more efficient.
I think you should start using CGI combined with one of those non-free database systems (I forgot how they're called though).
Sure it would mean recoding of the entire site, but it would result in much cheaper bills.

At 3/29/05 07:51 PM, F-13 wrote: Sorry to bump this topic, but I'd just like to say that maybe if NG was using a more reliable database system than mySQL, it would probably be much faster and much more efficient.
I think you should start using CGI combined with one of those non-free database systems (I forgot how they're called though).
Sure it would mean recoding of the entire site, but it would result in much cheaper bills.

Why use CGI? PHP isn't limited to mySQL, so if they were going to switch database software (which I doubt since it means $$ for both the software and the server cluster,) why not just leave the scripts exactly as they are and rewrite the database class?

Although, as unlikely as it may be in the near future, I assume that they'd be looking at commercial level db software for the future. mySQL has it's limits and NG is already pushing them. NG isn't getting any smaller, and mySQL certainly isn't in any hurry to get much better.