New Fiddle: Eggratings
When looking at products on Newegg, I’ve often wanted to know not just the rating of an individual product, but the average rating received by all products from that brand.
Computer technicians often have their favorite brands, and they will vary from person to person, usually based on individual experience. Even I was swayed by personal experiences.
I hope that with this script, a more objective qualitative measurement of component manufacturers will be available.
You can find it here, and expect to see more in-depth exploration into the code as I find time to write posts.
The server source code is up on GitHub, so head over there if you’d like to take a look at it.
For the “client” stuff (the user script) the script itself can be downloaded from the eggratings page linked above, and viewed in the same manner as any .js file.
Starting with the server-side stuff, a little searching revealed an article on the blog A Little Off. You can read in more detail on that article, but basically, he discovered that Newegg has a JSON system for use with their mobile app, that when properly accessed, functions similarly to an API.
In short, the server is taking in a brand ID (passed by Get variable, in this case from the user script) with the goal of returning an average user review rating for that brand.
Everything you need for dealing with Newegg’s data is in the EggFetcher class, so I’ll just go over that one. All of the other files are fairly specific to the particular way that I implemented it, and generally handlers for things like databases that are irrelevant to this particular topic of explanation.
There are four major functions:
GetProductsByBrand is the primary function, and will do as it says. Given a brand id, the function will return a list of products.
Since Newegg doesn’t really have an equivalent function (if you specify brand ID in your search parameter, you get one set of about the first thirty results, and can’t query for more). As a workaround, I took the response information from the brand query, which includes a list of categories that manufacturer has products in, and queried based on those categories (which will allow you to pull a full set of results).
This is where GetBrandCats, GetProducts, and GetProductsFromCategory come in.
GetBrandCats will get all of the categories for a given brand. The parameter that it asks for is a navigationItemList, which is a part of the JSON object returned when querying a brand. The function will decipher that particular portion of the JSON return, and pull the provided categories from it. The categories are then looped through and fed to…
GetProductsFromCategory will take categories one at a time, and loop through all of the pages that return for a given brand and category search filtration.
Finally, GetProducts will take the resulting JSON products and convert them, one at a time, into a custom product class used by the project. This cuts out any irrelevant information, and makes things generally easier to work with.
All of these products are added to the database, and the average value for a brand is found by simply selecting the appropriate rows, and having MySQL calculate the averages of the columns. A little simple math later, and badda-bing badda-boom, we have a simple number to return to the user for display.
The reason for dropping the database in the middle of this is caching. As you might imagine, making any number of requests to Newegg takes time. A couple seconds, usually, for an entire manufacturer catalog. This way, if we need to make a repeat request for a user, we have all of the information from Newegg stored locally, so retrieval will be much faster.