|
|
| |
Kategorier
|
|
| |
Blandat
(3)
Sånt som inte hör till någon annan kategori.
Datorrelaterat
(32)
Lite blandat tips och guider.
Filmklipp
(14)
Inte så mycket text, mestadels YouTube-klipp.
Nyheter
(3)
Diverse nyheter, mestadels kommentarer till en artiklar.
|
|
|
|
|
| |
Extracting search words from Apache log - Datorrelaterat - 2011-12-25 22:16:20
|
|
| |
Introduction
I'm doing a project right now were focus is on making the website as attractive as possible for Google.
I just made a couple of scripts that will parse the access log of apache and retrieve keywords used in referring Google searches. For example if I find "some search string" in the access log, I can dynamically create the alias bjurr.se/some_search_string.
The script
parselog.php
<?
print("Read content of log\n");
ob_start();
readfile("/var/log/apache2/bjurr.se.access.log");
$content=ob_get_clean();
print("Get words:\n");
$words = array();
$lines = preg_split('/\n/', $content);
foreach ($lines as $line) {
if (preg_match('/google(.se|.com)(.*)/', $line, $matches)) {
//print_r($matches);
if (preg_match('/q=([^&]*)/', $matches[2], $matches)) {
$match = urldecode($matches[1]);
if ($match != "") {
print("> ".$match."\n");
$words[$match] = $match;
}
}
}
}
//Connecto to db
mysql_connect("localhost","snylta","snyltaab12cd");
mysql_select_db("snylta");
foreach ($words as $key => $word) {
if ($word != "") {
print "Adding: ".$word."\n";
addWord($word);
}
}
function addWord($word) {
$sql = "INSERT INTO search_words (version, word) VALUES ('0','".addslashes($word)."')";
//print("$sql\n");
mysql_query($sql);
}
?>
|
|
usage
$ php parselog.php
Read content of log
Get words:
> postorter
> sdl tutorial
> installera svn
> binance
> why runtime.exec() hangs
> grails bjurr
Adding: postorter
Adding: sdl tutorial
Adding: installera svn
Adding: binance
Adding: why runtime.exec() hangs
Adding: grails bjurr
|
| |
|
| |
| |
|
|
|
|
|
|
| |
GIT over HTTP on Ubuntu - Datorrelaterat - 2011-12-25 11:28:38
|
|
| |
Introduction
I just installed GIT over HTTP on my server. And I'd like to share some notes of how I did it on my Ubuntu desktop installation.
Enable modules
I needed to enable two modules:
sudo a2enmod dav
sudo a2enmod a2enmod dav_fs
|
|
Add virtual host
sudo nano -w /etc/apache2/sites-enabled/000-default
|
|
<VirtualHost *:80>
ServerName git.your.domain
DocumentRoot /www/sites/git
<Directory /www/sites/git/>
DAV On
Options ExecCGI FollowSymLinks Indexes
# Deny everyything here
Deny from all
AuthType Basic
AuthName "git repository"
AuthUserFile /www/sites/git/htpasswd.git
AuthGroupFile /www/sites/git/htgroup.git
</Directory>
<Directory /www/sites/git/project>
Allow from all
Order allow,deny
<Limit GET>
Require group project-read
</Limit>
<Limit GET PUT POST DELETE PROPPATCH MKCOL COPY MOVE LOCK UNLOCK>
Require group project-write
</Limit>
</Directory>
</VirtualHost>
|
|
Setup GIT repository
mkdir -p /www/sites/git/project
cd /www/sites/git/project
git --bare init
chown -R www-data:www-data /www/sites/git/project
sudo -u apache git update-server-info
|
|
Clone the repository
Now locally ju need to do something like this.
cd ~/workspace
git clone http://git.your.domain/project
touch placeholder.txt
git add placeholder.txt
git commit -m "first commit"
git push origin master
|
| |
|
| |
| |
|
|
|
|
|
|
| |
Grails, Tomcat6, Apache2 Caching - Datorrelaterat - 2011-12-24 17:43:37
|
|
| |
Introduction
I've been struggling a while with getting caching to work. I'm using Tomcat6 and Apache2 and running an application built with Grails.
Apache2 Modules
I am using 2 modules. I initially tried mem_cache but is now using disk_cache because it is said to be of higher quality.
sudo a2enmod expires
sudo a2enmod disk_cache
|
|
Configuring mods
disk_cache config (disk_cache.conf).
<IfModule mod_disk_cache.c>
CacheRoot /var/cache/apache2/mod_disk_cache
CacheEnable disk /
CacheDirLevels 5
CacheDirLength 3
</IfModule>
|
|
The virtual host looks like this.
<VirtualHost *:80>
ServerName htmlunitgenerator.bjurr.se
ServerAlias www.htmlunitgenerator.bjurr.se
ProxyPass / http://htmlunitgenerator.bjurr.se:8080/
ProxyPassReverse / http://htmlunitgenerator.bjurr.se:8080/
CustomLog /var/log/apache2/htmlunitgenerator.bjurr.se.access.log combined
#Cache config
CacheStorePrivate On #Cache even if browser indicates it is a private site
CacheIgnoreCacheControl On
ExpiresActive On
ExpiresByType text/html "access plus 6 hours"
ExpiresByType text/txt "access plus 6 hours"
</VirtualHost>
|
|
What to cache
Only GET-requests are cached! So in Grails you need to do method="GET" like this:
<g:remoteLink action="show" id="1" method="get">Test 1</g:remoteLink>
|
|
If you want to cache other content types, here are some you may want to use:
ExpiresByType text/css "access plus 6 hours"
ExpiresByType application/x-javascript "access plus 6 hours"
ExpiresByType application/javascript "access plus 6 hours"
ExpiresByType text/javascript "access plus 6 hours"
ExpiresByType text/html "access plus 6 hours"
ExpiresByType text/plain "access plus 6 hours"
ExpiresByType text/xsd "access plus 6 hours"
ExpiresByType text/xsl "access plus 6 hours"
ExpiresByType text/xml "access plus 6 hours"
ExpiresByType application/java "access plus 6 hours"
ExpiresByType image/gif "access plus 6 hours"
ExpiresByType application/x-gzip "access plus 6 hours"
ExpiresByType image/x-icon "access plus 6 hours"
ExpiresByType image/jpeg "access plus 6 hours"
ExpiresByType application/pdf "access plus 6 hours"
ExpiresByType image/png "access plus 6 hours"
|
| |
|
| |
| |
|
|
|
|
|
|
| |
HTMLUnitGenerator - Datorrelaterat - 2011-04-24 09:59:07
|
|
| |
Introduction
I strongly support software testing. If you don't produce test cases that cover the requirements you implement, then you are not developing anything. It is impossible to reach sustainability, produce new functionality fast and with high quality or even cooperate with other developers, unless you work with automated test cases!
Being a web developer I've been faced with problem of testing front end of web applications. When it comes to GUI and making sure its look and feel is as it should, the testing is done by a human resource. But when it comes to making sure flows can be executed without JavaScripts, CSS or backend crashes, or just making sure a text actually appears on a webpage, I want automated testing.
HTMLUnit
HTMLUnit is a really nice tool. It is a head less web browser. This is perfect for creating test cases that clicks on different elements and does asserts on, for example, different attributes and values.
The problem is, we don't want to write those test cases in Java. Java is a general purpose programming language. The test cases will get messy and hard to read. They don't have to but the chances are they will.
Another problem is documentation of test cases. If we write test cases in Java we need to document them to make sure what flows are covered. This is extra important with front end testing, since these test can be very time consuming when testing a large web site.
HTMLUnitGenrerator
I created an open source project to deal with these problems. I have created a domain specific language, DSL. You may say the test cases are automatically generated from the documentation, since the DSL is so easy to read and understand, it could just as well be documentation to a test case written in Java.
Here is a sneak peak, check out GitHub for more examples.
See testcases/BBBSeePaths.flow
See testcases/BBBSeeUrls.flow
Go to baspaket and wait 2 seconds
Find a with attribute href set to /servlet/orderflow/search/search-flow?Id=tcm:142-23371 in campaignmodule
Click on campaignModuleChoose and wait 10 seconds
Find input with attribute id set to _eventId_search in searchpopup
Fill in locationForm with _eventId as search and phoneNumber.fullNumber as 0768966787
Click on _eventId_search and wait 10 seconds
Find input with attribute id set to _eventId_search in searchpopup
Fill in locationForm with address.floor as 3
Click on _eventId_search and wait 10 seconds
Find a with attribute href set to /orderflow/index.html?Id=tcm:142-23381&fromSearch&page=new in searchpopup
|
| |
|
| |
| |
|
|
|
|
|
|
| |
GIT setup - Datorrelaterat - 2010-08-12 18:03:52
|
|
| |
Introduction
After a few years of CVS and Subversion I thought it was about time to try this new cool thing called GIT. It was pretty easy to install and get started with... but I'd like to share some quick facts, and also supply a script that will make it easier to create new projects.
CVS/Subversion and GIT comparison
I'd like to quickly point out 2 differences in how you should think about GIT compared to CVS/Subversion.
1. In GIT you checkout, and commit. Just like CVS/Subversion. But to push anything to the server you also need to do push. And also, you do pull instead of update.
2. In CVS/Subversion you typically create one root directory and then you host alot of projects in sub directories. This is not how you should do it in GIT. You should create a new repository for each project. This means you will be creating alot of projects, I created a script to ease this step.
Installing GIT
See the links to the right for more details.
Do this on your client.
apt-get -y install git-core
ssh-keygen -t rsa
scp .ssh/id_rsa.pub username@yourserver.com:/tmp
|
|
Do this on your server.
apt-get -y install git-core gitosis
sudo -H -u gitosis gitosis-init < /tmp/id_rsa.pub
|
|
Creating GIT repositories
The script will give you the opportunity of editing gitosis.conf. You should add some content here. I'll paste an example.
[group binance]
writable = binance
members = bjerre@bjerre-laptop
|
|
In this example binance is the project name. bjerre@bjerre-laptop is a member. In this case there is only one member, but you can add more and seperate them with witespace. For this to work the public key of user bjerre@bjerre-laptop has to be present in gitosis keydir. In my case that is: /srv /gitosis /repositories /gitosis-admin.git /gitosis-export /keydir /bjerre@bjerre-laptop.pub.
The script
#!/bin/bash
#
# This script will create a new GIT project with a given
# name on a given server
# Author: tomasbjerre at yahoo dot com
#
#Store input
if [ $# -ne 2 ]
then
echo "usage: $0 project server"
exit
fi
PROJECTNAME=$1
SERVER_IP_OR_HOSTNAME=$2
echo Creating $PROJECTNAME on $SERVER_IP_OR_HOSTNAME
#Find an empty directory to use as sandox
GITCREATORSANDBOX=/tmp/`rand`
while [ -e $GITCREATORSANDBOX ]; do
GITCREATORSANDBOX=/tmp/`rand`
done;
cd $GITCREATORSANDBOX
echo Working in $GITCREATORSANDBOX
#Checkout the gitosis config, let the user add the project
git clone gitosis@$SERVER_IP_OR_HOSTNAME:gitosis-admin.git
cd gitosis-admin
nano -w gitosis.conf
git commit -a
git push
cd ..
#Create the project
mkdir $PROJECTNAME
cd $PROJECTNAME
git init
touch placeholder.txt
git add .
git commit -a -m "So it begins"
git remote add origin gitosis@$SERVER_IP_OR_HOSTNAME:$PROJECTNAME.git
git push origin master
cd ..
rm -rf $PROJECTNAME
#Remove the sandbox
rm -rf $GITCREATORSANDBOX
|
|
And finally, to check out projects you do this.
git clone gitosis@HOST_OR_IP:REPOSITORY_NAME.git
|
| |
|
| |
| |
|
|
|
|
|
|
| |
Introducing Binance - Datorrelaterat - 2010-07-10 10:37:55
|
|
| |
Introduction
I believe a lot of people, who are active at the stock market, are using spreadsheet applications such as Excel. People want to keep track of their past investments as well as monitoring their current.
If you have a highly diversified portfolio, say about 50 corporations, you will need to spend a lot of time updating you spread sheet. This project aims at eliminating the need of such a spread sheet.
Implementation
The software is implemented usign Groove and Grails in the front end. Pure Java is used in the back end (for retrieving stock data). A MySQL database is used for storing user data as well as current stock status.
I really like this project, so please contribute! I have very limited time to work on it myself.
More info
You will find the project home page at: http://code.google.com/p/binance/
Check out the code here: svn checkout http://binance.googlecode.com/svn/trunk/ binance-read-only
|
| |
| |
|
|
|
|
|
|
| |
Downloading MP3 from Youtube in Ubuntu Linux - Datorrelaterat - 2010-06-27 09:32:10
|
|
| |
Introduction
In a previous post I published software with GUI enabling download of MP3 from any YouTube video. However YouTube changes and I don't have time to manage. So I'd like to make a quick post on how I do it now.
The Script
You will need to install a few other programs to get the script running.
sudo apt-get install rand youtube-dl ffmpeg pacpl
|
|
Create ~/bin/ydl.sh, chmod it to 755 and put this inside.
#!/bin/sh
url=$1
filename=$2
tmpfile=`rand`
echo Downloading $url to $filename using temp-file $tmpfile
youtube-dl -b -o "/tmp/$tmpfile.flv" "$url"
pacpl --to mp3 /tmp/$tmpfile.flv
rm /tmp/$tmpfile.flv
mv /tmp/$tmpfile.mp3 /home/bjerre/Hämtningar/$filename.mp3
|
|
Ofcourse you will have to change "/home/bjerre/Hämtningar/"...
Now you can do something like.
ydl.sh "http://www.youtube.com/watch?v=8QSgNM9yNjo" "lena_-_satelite"
|
| |
|
| |
| |
|
|
|
|
|
|
| |
Observe field in Grails using Prototype - Datorrelaterat - 2010-03-21 10:31:54
|
|
| |
Introduction
Being familiar with Ruby on Rails I can not help comparing Rails with Grails. Today I was looking for a Grails method equivalent to the Rails method observe_field. I must say I was surprised with what I found.
Grails has a method called remoteField. This is as close as I get to Rails observe_field after a few minutes on Google. But this is far from an equivalent method. A very important difference is that observe_field will periodically check a fields value and fire an AJAX request if the value has been altered. While remoteField will use the onChange event and fire an AJAX request whenever onChange occurs. The later is a terrible solution, it does not ensure that the content of the field corresponds with the last response from the server. Neither will observe_field but it will detect such situations and correct them. In my case I have a search field and a div containing results, the results must correspond to the text in the search field.
Observer tag
Here is my solution =)
grails create-tag-lib project.tag.observe
|
|
Go to ObserveTagLib.groove and put this inside.
package binance.tag
class ObserveTagLib {
def observe = { attrs ->
def controller = attrs.controller
def action = attrs.action
def subject = attrs.subject
def update = attrs.update
def reference = attrs.reference
out << "<script language='javascript'>"
out << "new PeriodicalExecuter(function() {"
out << "if (\$(\'$reference\') == null && \$(\'${subject}\').value != '' || \$(\'$reference\') != null && \$(\'${subject}\').value != \$(\'$reference\').value) {\n"
out << remoteFunction(controller:controller,
action:action,
update:update,
params:"'$subject='+\$('$subject').value"
)
out << "}\n"
out << "}, 0.5);"
out << "</script>"
}
}
|
|
You can now do something like this in your gsp-views.
<g:observe
subject="query"
reference="query_reference"
controller="search"
action="results"
update="search_results" />
|
|
And somewhere inside the div referenced by the update attribute, in my case "search_results", put something like this.
<input
type='hidden'
id='query_reference'
value='${query}'/>
|
|
Where ${query} is the actual search query that generated the result.
|
| |
| |
|
|
|
|
|
|
|
|
|
|
|
|
