setrtoyou.blogg.se - Python ghost browser

#PYTHON GHOST BROWSER SOFTWARE#

The counter type is really useful for these situations.

collections - This module implements specialized container datatypes providing alternatives to Python’s general purpose built-in containers, dict, list, set, and tuple.

fileinput - This module implements a helper class and functions to quickly write a loop over standard input or a list of files.

To parse the logfiles and produce this goodness I wrote a little python script. But as soon as someone logs in, i can blacklist the ip and user-agent combination to disregard their traffic. Because of floating IP addresses i can't tell these users apart. But for now, 4 hours will do.īecause we have a few editors working constantly on these sites, we want to remove them from the statistics. Depending on the content, the 4 hours might be tuned, so I'm still figuring out what that number should be exactly. If you continue reading at 23:45 and continue to 00:03 you end up with three sessions. That means that if you open the browser at morning and read some pages on your cell phone, and continue reading during lunch, that you would probably have 2 sessions. To differentiate, i used the term "session".Ī session is a bunch of consecutive hits from the same ip and user-agent, bounded by inactivity of at least 4 hours.

I want SimpleAnalytics compatible statistics but with a visit that takes actual user behavior into consideration. So i wanted to fine-tune the log parsing a little. Furthermore, people can visit the same site at different times a day, which might be different visits (or sessions) to me. Given this definition you end up with two visits for one actual session if this crosses the date border. HTTP requests with the same IP, date, and user agent are considered a unique visit. But as a logparser GoAccess defines visits in the following way:Ī hit is a request (line in the access log), e.g., 10 requests = 10 hits. In earlier attempts to find a good web log analyzer i came across GoAccess which i really like, but didn't have support for caddy logfiles then, if i remember correctly. (While writing this article I discovered it's probably the Caddy logformat mentioned below).

#PYTHON GHOST BROWSER SOFTWARE#

For each request the software appends a single line with a complete JSON document to the logfile. Ghost doesn't log in the ordinary access.log formats. This is a metric which has no standard in logparsing software.Īs we ghost websites, there's a little catch to logging. I really appreciate what they have built, but it lacks one feature which they purposefully will not support because of their privacy-first principle: how long a user has browsed a website. Very importantly they don't show an average but give the median įurthermore, because they host it, i don't have to build and host all of that goodness myself. They discard bots, they actually adhere to the the do-not-track header and to keep stats a little realistic, they disregard browsing activity that was less than 5 seconds to avoid page-bounces. It's privacy focus from the ground up and therefor doesn't require a cookie banner. They provide a very intuitive analytics page which our customers appreciate.

At EducationWarehouse we use SimpleAnalytics these days to monitor our website usage.