Category: General

PyData conference in Barcelona

pydatabcn2017I was lucky to attend PyData conference in Barcelona this year, hosted in ESADE.

Although I’m basically a PHP developer, I’ve been playing with data science tools lately with python’s stack. I have no real experience in data science, apart from a couple of prediction coding using linear regression, but I was curious.

With a novice spirit, I set some clear objectives: find out if data science is like teenager sex, or companies are really using it; get a feeling of the community; and try to learn as much as I could.

First of all, the community is vibrant, actually far more than PHP’s one in Barcelona. The organization was smooth too, and all the people I talked with was really nice. Everybody had things to learn, so came with an open mind.

It was funny to see that I was on the “data owners” side, while most people were in the “looking for datasets” side. This led to several conversations asking me how we use the data in our company.

Regarding the talks, there were quite a lot about tools. Python science stack have a wide range of evolving tools, and this somehow reminds me of PHP circa 2008, when basic tools (PHPUnit, for example) were becoming popular. It’s good to polish your tools and master them, so I welcomed those talks.

There were also some talks on theory, which surprised me, as I haven’t never seen university professors in software conferences. Mathematical and computer science concepts were explained, for instance on optimization. This contrasts with the common industry solution: if some code is slow, just use more machine instances, which is far cheaper that spend time trying to optimize things (at least 99% of the time). I don’t mean I didn’t like those talks (actually one was really mind blowing), but I would love to see more professors in some other conferences, getting a real feel of some industry practices.

I was looking for talks showing “real fire”, real examples in companies. We heard about hotels trying to predict cancellations (in order to do overbooking); we saw IBM’s Watson analyzing the personality of customers; predict which employees will leave a big company; ideas to react knowing bad weather will arrive; best weekday to publish job offers and set interviews; and some other extremely interesting stuff… but I do want more!

My overall feeling is that I learned a lot. Python is not really used as a language but more as an interface for some amazing libraries. It looks like I have no option but to start exploring the data in ulabox!

I’d like to thank ulabox (my employer) that paid the ticket, and all the people in the organization that did a great job!

I published some of my (unedited) notes too.


General programming principles

This is just a list about programming principles that I’m making for myself. These should be instinctive to any developer.


Good password trick

· You shouldn’t use the same password in different places.
· You should change your password from time to time.

Despite these 2 basic rules, most people has just 2 or 3 passwords, and use them everywhere. So I’d like to share the following trick, that I heard from my former boss:

good password = your usual password + first character of the website/place’s name + last digit of the year

For example: if your usual password is “sadshoes”, and you are setting it on facebook, your new password should be “sadshoesf3″ (“f” from facebook and “3” from 2013).

This will be far more stronger than just “sadshoes”, and if somebody breaks your password, he will only be able to enter in facebook. This will also force you to change your password every year.

Other stuff about passwords:
· Do not think that somebody will try “manually” to break your password; nowadays a bot will try to break your password using a long list of common words used as passwords.
· Never use pure words, or concatenated words, alone: bots also try these.
· Moreover using common text-to-digit translations, like “LOVE” to “L0V3″, is not secure: bots also try these!


10 years to master

You need 10 years to become an expert on a field. No more, no less. 10 years of continuous effort. Think about a famous musician, a famous athlete or even a professional Go player. There is at least 10 years of hard-work in almost all cases before he becomes a star.

Looking at this video by Danny MacAskill, I imagine those 10 years he spent improving his technique!

When you start with a subject, you experience different statuses, from novice to expert. But there is a key point that only an expert can do: he makes it look easy. As if you can take your bike and do same things showed in the video. Actually I could say I have 8 years of experience on Go, and perhaps 10 years on PHP, but I feel there is still a lot to learn. Despite sometimes people see that expert magic on me.

I wonder which new expert skills I’ll have in 10 years. Things that you start now doing often can become key points of your life. Things that perhaps you don’t really consider, like gardening or doing DIY. So, be sure to avoid wasting your time, and focus on practicing interesting skills. We’ll see what happens in 10 years.


FriendFeed first impressions

In my previous post I criticized Facebook (“Fb”) quite a lot. But at the end it does some good stuff, like the “keep in touch with your people” thing, a decent content aggregator and a way to publish (or share) things like personal mood or thoughts, links, images, etc. But if you distill some of these good parts of Facebook, and improve them, you get FriendFeed (“ff”).

FriendFeed is a content aggregator which is far more complete than the one in Facebook (compare just the numbers: on Fb there are 13 possible sources to aggregate from, and 57 in ff). Actually Fb aggregator is quite bad, because some sources are not updated, and you need a 3rd part app to do so. ff has also a way to share links, images and such, and it’s really impressive (just add its Bookmarklet in your bookmarks bar and see its power). Moreover you get the usual “publish what are you doing”, “like it” and “make a comment” actions.

But the most pleasant thing I found is that it is a real Web 2.0 site. You add content, everybody see it, and there is RSS. Try to search for RSS in Facebook, find nothing. You can even get a widget (as you see on the right column, if you are visiting www.liopic.com). The groups (called “rooms”) are meaningfulness, with current people interested in the subject of the room.

Anyway there are 2 things that I miss from Fb that I’d like to see in ff. First is “events”, a way to invite people to an event and mark dates (despite there are complete websites that do just so). Second is “amount of people”… but this is a question of time: first time I entered in Facebook nobody was there, a year later a lot of people signed up. Let’s hope same thing will happen in FriendFeed.

So, see you there.


Facebook, or waste your time channel surfing

Have you ever feel how fast the time goes by when you do some channel surfing on TV? “I spend 2 hours in front of the TV, changing from one channel to another, and finally I’ve not seen anything at all”. I had this thought of “wasted time” while surfing on Facebook recently.

Fb, the origins
I logged in on Facebook some months ago, but I didn’t understand the meaning of the website at first. I came from social websites that focus on one thing, like Flickr (pictures) or Last.fm (music listened), so it was a bit weird to see a social website which end is the social relation itself, without a “theme”. Moreover the only friends you are supposed to have there (and to let them see your profile) are just the friends you currently have in Real Life, so basically Facebook aims to be a copy of Real Life society. So don’t expect new friends, just expect to see what your current Real Life friends are doing… but basically you can do so with other methods, like IM. I mean, when I use Internet, I don’t expect to see a mirror of Real Life, I expect to see different things, improved things… or just surf Wikipedia.

Fb, closed to foreigners
Another surprising thing is that your profile is private, a random net user can’t see it, unless he is your friend. This “closed” philosophy as default annoyed me. It could make sense, to keep your data from public. But coming from websites where you add content, mostly public (like photos in Flickr), and let the world see your stuff, this philosophy is at least paranoiac. Of course you can mark pictures as private in Flickr, and you can open your Facebook profile to everybody, but how many people do so? How many know how to do so? The default settings are the ones that show the website philosophy.

Fb, the false sense of security
Despite it claims for keeping your stuff away from foreign eyes, there are a lot of back-doors to overpass the private settings. The easiest one is just see Mr.X friends list, and look for one with open/visible profile, let’s say Ms.Y… then you can see pictures tagged with Ms. Y, even from the Mr.X picture sets… finally you get full access to Mr.X pictures. Another way is creating an application, using Facebook API: when somebody uses your fancy app, you can get FULL access to all his data, and can (for example) save it in your personal server. So you only need to create a small test app (“How freak are you?”) to harvest a lot of private data.

Fb, trying to create content
Of course in a social website without theme there will be less content created, and user will come back less often. Therefore Facebook staff have to help people to produce content, in order to induce other people to revisite the web often. One way is acting as an “aggregator” (gets content from other web2.0 places, like Flickr or del.icio.us, and publish again in user’s page). Other is creating an API to create applications. 3rd part developers come to create apps, salivating by the amount of possible users for their apps, and Facebook is happy to see more stuff in his system. I created a couple of Facebook apps and I can claim that their API is a torture. Basically they don’t let you use libraries, like jQuery, so you are forced to write old style code. Later you discover you can ignore it and create an iframe for your app… so that means your app is not a real Facebook app, just a nice Facebook frame that borders you website in your personal server.

Fb, a waste of time
So, I can guess that the 80~90% of produced or pseudo-produced content in your profile (or friends profile) is low quality. Do you really need to know that your old friend from school Harry loves chocolate? Nah!

Of course Facebook has good things: the content aggregator and a way to publish things neither short nor long in a fast way… but for such things you can use a better website, like Friendfeed.


Sixth Sense

In my previous work I enjoyed when some coworkers tried “visual things” using a webcamera and a soft to recognize hands, fingers and such. This morning I saw a video with the last research on the field… impressive:

Link to the video

Which technology will win in the input world? Cameras or multi-touch screens?


You can even find “plants 2.0″

Laurel with new leaves While my plants grow and grow, and are eager to the arrive of summer, RubĂ©n told me about a website, a la 2.0, to keep track of your gardens and plants: My folia, where I’ve created my garden profile. There are too few people at the moment, so the magic of 2.0 doesn’t work still. And I’m not sure about their business plan (if they didn’t build it for fun).

Anyway I found it interesting as a place where I can investigate the growing process of a 2.0 website. I’m preparing the launch of my own website 2.0, focused on strategy turn-based games, so every single detail I can observe in other places could be interesting.


A PDA with Linux

I’m used to read PDFs while in bed, on my Acer PDA. But its tiny resolution (320×240) didn’t help a clear reading, therefore I started to look for a bigger device, it must be easy to hold but not too big. So I avoided those e-readers, and focused on PDAs with bigger resolution. And then I stumbled upon a Nokia 770 review… it is supposed to be a “Internet tablet”, but the thing that caught my attention was its screen resolution: 800×480. Luckily I have a friend who owns one, so I had a look at the device… it was interesting indeed.

I thought “Ok, I can use a N770 to read PDFs, and my Acer n50 for the rest of things (do sudokus, see pictures and record Go games)”… because I read some comments on Internet saying “it’s just a web browser, not a really PDA” (really?). I found a offer on ebay, and bought it for just 102 euros.

What a machine! The default factory configuration was a bit short of features (I reinstalled the lastest OS), but then you configure some program repositories, start to add open source applications, and discover the real power. The OS is a version of Debian. This means you can have an xterm with most of the usual commands. Moreover the graphical system can hold GTK-based programs with an easy porting. I installed some games, CGoban to record Go games, and some utilities (including a Last.fm player).

The cons are that its CPU (252 MHz) is too slow, so sometimes it stops to respond for a while; and the battery lasts less than a normal PDA (actually, similar to a laptop, around 3~4 hours). So its a device for patient users (or if not, try its successor, the N800 and N810, with double speed CPU and webcam).

I can even play a game of Freeciv!
Playing Freeciv on a Nokia 770


Renfe.es: a bad UI case

In the last year I flew 24 times. Too many times, according to those ecological footprint quizs. So I decided to give another try to the train, for my usual Barcelona-Alicante trip. Unluckily in Spain the train’s company, “Renfe”, is really prehistoric: there are always delays, tickets are expensive (domestic flights are cheaper and quicker), and is the focus of a lot of political scandals.

Last Christmas they announced a new web, with a new modern system of buying tickets, and discounts up to 60% in the normal price. So far, the reservation system was terribly bad. Can they evolve into something “normal”?

I tried to buy a return ticket, Barcelona-Alicante. Firstly, the old fares are still there. Actually there is a new thing, a Captcha that gives you access to the classical fixed prices… whenever you want to do your trip, the prices are always the same for the same trains, so why do they need to hide them under a Captcha?

But the best part is to arrive… If you want to get access to the new “web fare” (60% off), you must choose “One way” ticket at the first form. This means: you must do the process 2 times, one for your going ticket, and one for the return ticket. Later you have to find a small combo box that let you choose among the fare list… the default is the most expensive option.

Finally a frame with you bank web is opened, to verify you credit card details. You press “Ok”, and suddenly Renfe.es says “this train is not available”… at the end of all the process. Monkeys do better jobs.

After 2 hours I accomplished the mission, I got my ticket with a cheap price. But I suspect that it has been deliberately implemented in this weird system. I imagine people, elders, in the marketing department of Renfe, smiling at the time they say “hehe, we have offered a modern look, but we are not going to let anybody a cheap price ticket”.