Quantcast
Channel: geekyprimitives

Spring Integration with JGroups

0
0

This time something different, a framework :)

From time to time, even I, professional framework “hater”, need to write or contribute to some framework. Just to prove how “bad, bad, bad” frameworks are :) Just to feel “filthy” and “dirty”. Just to feel this chill down my spine, when I have to frame my thoughts within boundaries set by somebody else:)

This time I was playing with Spring Integration, an interesting approach to implementation of patterns described in a book
“Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions” by Greg Hohpe and Bobby Woolf, a real classic.

As I was playing with different adapters and channels I have noticed that there is missing implementation of JGroups transport.
So here is my short story how I have built channel adapters and XML namespace for JGroups in Spring Integration.

This small chunk of code is of course available for your eyes only and convenience at Github, https://github.com/kcrimson/spring-integration-jgroups.

If you are into Enterprise Integration Patterns, you probably know that Channel Adapter pattern allows you to access particular API or data using Message Channel semantics.Just like well known Adapter pattern from “GoF” book, it wraps one interface into another one, so you can access it without changing existing code.
Channel Adapter is one of key patterns, which allows you to “hide” complexity and different architectural styles or APIs, behind simple uniform interface of Message Channel.Thanks to this pattern you can easily access FTP sites, invoke HTTP based services and send JMS messages without diving too deep into complexities of each protocol or API, and at the same time you can easily replace FTP with email or HTTP transport, thanks to this pattern.

This is also simplest, first step, you can take when you want to implement your own transport/endpoint in Spring Integration. Of course in majority of cases, you will have to implement two Channel Adapters, one for inbound and one for outbound communication.

So let’s start with inbound channel adapter, which receives messages send to group of nodes (JGroups cluster). In the case of JGroups we deal with “push” communication.Out of many abstract endpoint implementations and styles available in Spring Integration, org.springframework.integration.endpoint.MessageProducerSupport looked like a most obvious choice.

It has some basic code for setting Message Channel reference, convenient lifecycle methods and as well handling of messaging errors. So what’s left to do? Put some glue code which registers receiver on JGroup’s JChannel object.

	@Override
	protected void doStart() {

		jgroupsChannel.setReceiver(new ReceiverAdapter() {

			@Override
			public void receive(Message msg) {
				Object object = msg.getObject();

				Map<String, Object> headers = headerMapper.toHeaders(msg);

				sendMessage(MessageBuilder.withPayload(object).copyHeaders(headers).build());
			}

		});

	}

Few commits later (including “crash course” on XML Schema), I was able to connect to JGroups cluster and send received JGroups messages to any channel, with a little bit of XML which looks like this.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:jgroups="http://www.springframework.org/schema/integration/jgroups"
	xmlns:int="http://www.springframework.org/schema/integration"
	xsi:schemaLocation="http://www.springframework.org/schema/integration/jgroups http://www.springframework.org/schema/integration/jgroups/spring-intergration-jgroups.xsd
		http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration-2.2.xsd
		http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">


	<jgroups:cluster name="mygroup">
		<jgroups:xml-configurator resource="classpath:udp.xml" />
	</jgroups:cluster>

	<jgroups:inbound-channel-adapter id="cluster-adapter" cluster="mygroup" channel="inbound"/>
	
	<int:channel id="inbound">
		<int:queue/>
	</int:channel>
	
</beans>

I hope you noticed, JGroups cluster name is a also name of a bean which we inject into inbound channel adapter, using attribute cluster

Outbound channel is also trivial piece of code, which extends org.springframework.integration.handler.AbstractMessageHandler. So if you want to send message from Spring Integration to JGroups cluster, you can simply

<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:jgroups="http://www.springframework.org/schema/integration/jgroups"
	xmlns:int="http://www.springframework.org/schema/integration"
	xsi:schemaLocation="http://www.springframework.org/schema/integration/jgroups http://www.springframework.org/schema/integration/jgroups/spring-intergration-jgroups.xsd
		http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration-2.2.xsd
		http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">


	<jgroups:cluster name="mygroup">
		<jgroups:xml-configurator resource="classpath:udp.xml" />
	</jgroups:cluster>

	<int:poller fixed-rate="100" default="true"/>

	<int:channel id="inbound">
		<int:queue/>
	</int:channel>

	<jgroups:outbound-channel-adapter id="cluster-adapter" cluster="mygroup" channel="inbound"/>
	
	
</beans>

This code is not complete, works only in really basic cases, but it already has basic header mapper, and as you saw some support code to start JGroups in Spring’s application context.
I welcome any thoughts, comments and suggestions and I am happy to maintain this code if you need it :) , good night.



“I ty też możesz mieć swoje dane w cache”

0
0

Tym razem w ojczystym języku, czyli moja prezentacja o cache w aplikacjach, dzięki uprzejmości i gościnności Polish JUG.

Stokrotne dzięki za możliwość prezentacji premierowego materiału, ciągle na etapie eksperymentów z formą i treścią :)

“Enjoy the ride!” :)


My “Practical Spring Data” accepted JavaDay conf in Riga

0
0

My “Practical Spring Data” accepted JavaDay conf in Riga

This is going to be a great time for me and I hope for attendees as well :) . I must admit I have never ever been in Riga and I have never ever been beyond east side of Polish border. I need to start looking for accommodation and flight. Any suggestions are more than welcomed :)

See you there in November.


Real World Enterprise Integration Patterns workshop at JDD conference in Krakow, 16-17 October 2013

Graphs, pyramids and organic growth

0
0

For last couple of months I have been thinking, researching, trying and failing to build different approach to software architecture.As I am getting closer to my 40ties it is high time to summarize what I have learned so far. All the past I spent following others, learning from people, adapting paradigms, testing them in real life, and throwing majority of them to a trashcan, as they were idealistic, utopian visions of reality.
Does it mean I want to concur the world with yet another manifesto? Give consultants a chance to write one more book? Does it mean I want to start a revolution? Ask you to forget all you have learned? For God’s sake, no. If I every do this, it means that I was drunk, or somebody forced me to do it.
The things I think about is rather a librarian like work, a village’s shaman who decided to put all the collective knowledge and wisdom of the tribe, written in the stone. It is rather collection of articles,blog posts, discussions, things so obvious they even don’t have name, so it is hard to talk about them.
So I named this thing, I called it Patterns of Organic Architecture. Nice name, would look nice at the cover book I will never have time to write.

What’s all about? Over the years I tried many approaches to software architecture. We, as an industry have written tons of worthless material around it, and we still struggle what does it really mean “to do software architecture”, “to be software architect”. We tried big design upfront, we tried bottom-up and top-down, we tried “don’t give a f..k about it” (so called Agile, or rather how we interpreted it). And yes this post is NOT against Agile, so stop whining about how bad I am, that I don’t understand it, I don’t believe in values. Again f..k it.
I am talking about reality here, not about Agile as an idea. I am talking about us, about us who had to implement what was said by Mr. Senior Principal Enterprise Architect, about us asked to implement one of these “world class” monolithic architecture frameworks, about us being asked if we can improve overall architecture, in the meantime adding some sexy features here and there. About us hearing everyday that there is no time for architecture, because we need to deliver some mythical business value, which usually turns into customers leaving product, because they had to wait to long to log in, because they had to wait to long to get new feature. Whatever you do to build your architecture, or even if you don’t care, you still can use what I call Patterns of Organic Architecture. A nature’s way of building simple and beautiful stuff.
Let it grow, go with the flow, but don’t leave it alone. Watch it. Measure it. Dig in, and when necessary take action. Understand forces that drive your system, and use them for your own good.
One of the patterns I have identified, “grow and harvest”, assumes that over time there will be pieces of your system which will become stable, by stable I mean ratio of changes in last months is close to zero.
One of the problems is how to identify these pieces, so we can “harvest” them from the code base, seal in separate repository, release binary artifact, or even remove them because they are not used, and enjoy smaller code base, faster build times and so on.
The idea is not new, this is something Michael Feathers is talking about quite often in his posts. Your SCM has all the information you need, you just need to dig in, reach out. The problem I have is that I tend to find myself in situations where I need to deal with gargantuan code bases, years of history, tons of files and technology and architecture changes, and if in majority of cases simple Perl script will do the job, it is hard, really hard to reason about such beasts without better tools.
Recently a friend of mine (to some known as @LAFK_pl) asked me for help with some interesting problem he was trying to solve with graph database, this way I found out that Neo4j was just updated to version 2.0. I was working with Neo4j couple of years back, trying to replace some legacy system for airlines with lovely graph model of airports and flight connections. Since then I didn’t had time to really track what is happening in this space, until last week. And suddenly I realized that graph is all I was looking for. What if I push all the information I have about files, Maven modules, packages and such plus SCM change sets in one graph, I can ask for anything that comes to my mind? Couple minutes later I had came up with this dirty code snippet which reads data from Mercurial log and puts content of this log into Neo4j database. Beware!!! This is not OOP, DDD, TDD code, this is just few minutes hack.

import static com.google.common.collect.FluentIterable.from;

import java.io.FileReader;
import java.util.List;

import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathFactory;

import net.sf.saxon.om.NodeInfo;

import org.neo4j.graphdb.Direction;
import org.neo4j.graphdb.DynamicLabel;
import org.neo4j.graphdb.DynamicRelationshipType;
import org.neo4j.graphdb.GraphDatabaseService;
import org.neo4j.graphdb.Node;
import org.neo4j.graphdb.Relationship;
import org.neo4j.graphdb.Transaction;
import org.neo4j.graphdb.factory.GraphDatabaseFactory;
import org.neo4j.graphdb.index.Index;
import org.neo4j.graphdb.index.IndexHits;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.uncommons.maths.combinatorics.CombinationGenerator;
import org.xml.sax.InputSource;

import com.google.common.base.Optional;
import com.google.common.base.Predicate;

public class App {

	private static final Logger LOGGER = LoggerFactory.getLogger(App.class);

	public static void main(String[] args) throws Exception {

		GraphDatabaseService graphDatabase = new GraphDatabaseFactory().newEmbeddedDatabaseBuilder("db")
		        .newGraphDatabase();

		XPathFactory factory = XPathFactory.newInstance();
		XPath xpath = factory.newXPath();

		@SuppressWarnings("unchecked")
		List<NodeInfo> changesets = (List<NodeInfo>) xpath.evaluate("/changes/changeset", new InputSource(
		        new FileReader("out.xml")), XPathConstants.NODESET);

		for (NodeInfo node : changesets) {

			@SuppressWarnings("unchecked")
			List<NodeInfo> files = (List<NodeInfo>) xpath.evaluate("file/text()", node, XPathConstants.NODESET);

			if (files.size() >= 2) {
				CombinationGenerator<NodeInfo> generator = new CombinationGenerator<NodeInfo>(files, 2);

				Transaction tx = graphDatabase.beginTx();
				Index<Node> nodes = graphDatabase.index().forNodes("files");
				for (List<NodeInfo> pair : generator) {

					NodeInfo first = pair.get(0);
					NodeInfo second = pair.get(1);

					LOGGER.info("changeset pair {}<->{}", first.getStringValue(), second.getStringValue());

					Node firstNode = addNode(graphDatabase, first, nodes);
					final Node secondNode = addNode(graphDatabase, second, nodes);

					createChangeset(firstNode, secondNode);

					createChangeset(secondNode, firstNode);
				}
				tx.success();
				tx.close();
			}

		}

	}

	private static void createChangeset(Node startNode, final Node endNode) {
		Iterable<Relationship> relationships = startNode.getRelationships(
		        DynamicRelationshipType.withName("changeset"), Direction.OUTGOING);

		Optional<Relationship> firstMatch = from(relationships).firstMatch(new Predicate<Relationship>() {

			public boolean apply(Relationship r) {
				return r.getEndNode().getId() == endNode.getId();
			}

		});

		Relationship relationship = firstMatch.orNull();
		if (relationship == null) {
			Relationship relationshipTo = startNode.createRelationshipTo(endNode,
			        DynamicRelationshipType.withName("changeset"));
			relationshipTo.setProperty("times", 1L);
		} else {
			Long property = (Long) relationship.getProperty("times");
			relationship.setProperty("times", ++property);
		}
	}

	private static Node addNode(GraphDatabaseService graphDatabase, NodeInfo df, Index<Node> nodes) {
		IndexHits<Node> indexHits = nodes.get("filename", df.getStringValue());
		Node single = indexHits.getSingle();
		if (single == null) {
			single = graphDatabase.createNode();
			single.addLabel(DynamicLabel.label(df.getStringValue()));
			nodes.putIfAbsent(single, "filename", df.getStringValue());
		}
		return single;
	}
}

This piece of code reads all change sets from the XML, and for each file, which in this case is a node in a graph, creates relation to another file in the same change set. If such relation already exists it just simply increases counter which is stored on the edges (relation) between nodes. Of course I am lazy enough to not write my own implementation of combinations from combinatorics, so I use Uncommons Math, and its CombinationGenerator, which generates pairs of all of the combinations of files within one change set. One thing I have spotted working later on with this graph is that, because Neo4j stores directed graphs, I had to generate two relations per each pair, incoming and outgoing. Which in fact is true, file A was changed together with file B, and file B was changed together with file A. Thanks to this I could simplify my Cypher queries. For those of you who are new to Neo4j, Cypher is a language which allows you to work with graphs, including queries and modifications. So what kind of information I can get from graph?

neo4j-sh (?)$ MATCH (a)-[c:`changeset`]->(b) RETURN labels(a),c.times,labels(b) order by c.times desc limit 5;                         
+-----------------------------------------------------------------------------------------------------------+                                                                
| labels(a)                                      | c.times | labels(b)                                      |                                                                
+-----------------------------------------------------------------------------------------------------------+                                                                
| [".../listeners/SummaryScenarioListener.java"] | 13      | [".../listeners/LoggingScenarioListener.java"] |                                                                
| [".../listeners/LoggingScenarioListener.java"] | 13      | [".../listeners/SummaryScenarioListener.java"] |                                                                
| ["pom.xml"]                                    | 12      | ["roadrunner-core/pom.xml]                     |        
| [".../cli/BenchTest.java"]                     | 12      | [".../cli/RunTest.java"]                       |                                                                
| [".../cli/RunTest.java"]                       | 12      | [".../cli/BenchTest.java"]                     |                                                                
+-----------------------------------------------------------------------------------------------------------+                                                                

This is the simplest query, which shows all pairs of files which where modified together, ordered by number of times this pair occurred in any change set. What is important to understand is that this kind of analysis not only shows code level dependencies, but as well feature/function level dependencies, it can also show cross technology dependencies, between your JavaScript, CSS and Java files, which is pretty hard to get even with all modern IDEs we have at our disposal.Of course this type of SCM “big data” :) analysis can be sometime misleading, there can and will be a lot of “falsy truths” about your code base. Especially when your teams favor large commits at the end of each sprint (which I hope doesn’t happen in any organization taking code quality and continuous integration seriously). But compare this kind of information with no information at all. At least you have places in the system from which you can start you travel back in time. Of course you use other tools to visualize your code, like Gephi or Graphviz. In my past I did many such “back in time” travels, and I was always coming back with interesting and precious information. In many cases as a result we were reorganizing code base, cutting of new libraries, Maven modules and so on. Better built times, less complex code. It is worth to look back. It is worth to “grow and harvest” you code. Enjoy!


In mind of 4Developers’2015

0
0

So, end of call for papers for 4Developers’2015 is approaching fast, it is actually end of January. I will be honest with you, I don’t have a clue what I will be talking about at “the only”, “the biggest” software architecture track in Poland, this year :). Yeah, I little bit of megalomania is never enough.
But actually it is true, in 2014 we had two full rooms of people, in parallel, listening to real world cases studies from battlefields.
Why I am writing this post at my almost dead blog? Simply, because a single tweet is not enough to tell you about my plans for this year. My dream is that every year, software architecture track at 4Developers will have different leitmotif. I also hope it will help you to find a topic you can talk about.

The word for this year is:

FAILURE

Failure in many different shapes. Tell us how and why you failed implementing newest, coolest architecture paradigm, how you design your systems, so they can fail in a controlled way. Tell us how you design for failure and how you fail fast.

Of course if you want to talk about something else, that’s cool, but I will prefer talks about failures, in any technology and framework, over other talks. So don’t wait to fail, submit your talk to 4Developers at CfP page, or write to me directly. In case you are not convinced you are a right person to stand in front of crowd to talk about failures, believe you are.


We are crowd, we are anonymous

0
0

WARNING
This post is full of emotions, strong words, my own opinions. It is far from being politically correct in any dimension. If you know me, you know what I mean. You read this post on your own responsibility.
I struggled since Friday two weeks ago to actually get it in shape and post it. I have compressed a year long of frustrations and anger.
Enjoy!

Friday, finally. When I was leaving office I was filled with anger, disappointment and this weird feeling of lost hope and faith in humanity. I am coder, by heart, and by accident I work with people.

On the way back home I was thinking about this post as a way for me to deal with all my frustration, being mainly driven by my interactions with other people. As I was passing tram stops I finally realized that there is something different which is a reason of my bad mood and what a surprise, the reason is different then I thought.

I think the way I was feeling that day is not only my problem, my dear angry, frustrated fellow coders, bytecode lovers, monad travellers, big data miners. As I travel through space and time, at conferences, code retreats, beer talks, JUGs and many other places I meet tired and frustrated coders. Frustrated for different reasons, constantly talking how codebases we work with suck, how stupid architects are (and yet majority of us wants to become one, to join this army of “codeless” monkeys, remember I work as architect at the moment :) ), how we hate trendsetters at conferences, how technology X sucks big time, and Y is new cool, and will suck in a year from now, for sure. Man, I have never met so many frustrated people as I meet at software conferences, we are so sad drinking energy drinks, looking for next sticker to put on our laptop and talking in corridors that last year this conference was much better. Still looking for Big Next Thing(TM).

Fuuuck, stop it. Right now.

Yeah, actually I used F word, and you know what? You can stop reading, I will not force you to continue. Use your free will or relax and fuck it (oops again).

Conference sucks? Don’t go.
Technology sucks? Don’t use.
Codebase sucks? Change it or leave it.
Your manager is an asshole, who doesn’t understand software? Become one.

I was thinking about all the bad emotions in my life as code mangler and system’s destroyer and maybe, just maybe I have an idea why it is so. Look, we are creators, we turn null terminated arrays of characters into electrical impulses, which create motion in resistors, capacitor and transistors, this motion of electrons changes people lives, wakes up good and bad emotions, helps people meet, get married and divorce, buy new shit, sell old shit, travel, get lost and be found.

Yes, it is you my friend, who created this mess. As I said at JDD this year, We are Gods!!! We own this planet!!!

… and yet we are crowd, we are anonymous.

We are like a wannabe rockstars, still rehearsing at parents garage with this notion, that we will never actualy record an album and play gig for a crowded house.

I think this is hidden root cause of all evil.

How many times you have heard from your customer, “I saw your recent change to registration form, from the first look I knew it was you who coded it. That’s so much better then previous one” or “I think this pagination on orders page was probably coded by Tomasz, it must have been him”. Yes we are anonymous, we create stuff, but we don’t get direct recognition, our code doesn’t hang in art galleries so everyone can see it and be amazed (or maybe it just sooo bad). I miss this feeling of direct connection between my creation and people who see it. I am a man behind wall of text, and you know what? Somebody could write different sequence of characters and nobody would notice. Especially true for all back-end people, for whom character, black and white terminal is an ultimate form of human thoughts manifestation.

And it will be always this way. Nobody will notice if your code is good or bad, if its reactive or declarative or imperative. Nobody will notice your fancy lambdas, sexy tricks with communicating sequential processes, and how many microseconds you saved due to use of sun.misc.Unsafe. Nobody.

Really?

That’s not true. Fortunately or not there are other software developers in the world, who will understand you techno mumble bumble, your 3 letter acronyms and beautiful details of endofunctors, monoids, actors and such. It is just simple as that, stand up and go to meet up or conference. Don’t tell me you need approval or training budget, this is just your stupid excuse for not moving your ass of the chair. Meetups are for free. Conferences?
Become a speaker! Give something to get something back. Stop complaining. Do something. Are you afraid to speak in public? Seriously? Do you know that all people who do it are afraid every time they go on stage. I am freaking scared every time, doesn’t matter if this new or old material, I am scared.

“People will eat me alive, if I say something stupid”. Seriously? If they will be right, because you actually said something stupid, that’s good for ya. You learn something. If they were trolls or morons, ignore them. And yes I am talking to you mister, in stupid baseball hat with more stickers on your laptop then successful pull requests in your whole life. Don’t wander around in corridors during conferences with this Mr. Knowitall face expression. Do you think you could do better on stage? Talking to crowd about your passion? Feel warmly welcomed. But just stop complaining and shut up. Majority of us speaking at conferences don’t get paid for it, we do it because we love it, show some respect.

“I don’t have anything interesting to say, I am doing JEE code maintanance”. Seriously? That’s awesome. Tell me about it, share your tricks and techniques. Every project is different and unique in a way it was fucked during design or development stage (and yes we do design and development phase, shoot me).

Don’t want to talk about it? It is even better. Find something. Try it. Fail. And share.

We are crowd, we are anonymous.

That’s the essence of why there is so much tension between us and the world. I am not complaining, really. I am just trying to point out how hard it is to take pride of our work. Surrounded by deadlines, messy code, procedures, processes and approvals, “agile”, “lean”, “risk assessment”, “business value”, end year reviews, improvement programs. Oh yes, do you love these moments when somebody non technical tells you that you need to improve your coding skills. It is like eunuch telling you how to make love with your wife.

All we create is “just a code”. It couldn’t be so hard? How many times you have heard this?

Or my other all time favourite “I used be a coder, too”. Wow, how fucking scientific, can you tell me why you don’t code any more? Is there any particular reason your rights to code repository were revoked?

I choose “a way of code”. Long time ago, I had my ups and downs. But this is my choice. And if you made same decision you need to accept that you need to find other ways to be proud of your work. And your day job is not necessary the only place. Conferences, meetups, hackathon, finding other freaks like you.

Just do me one favour, stop complaining, criticizing and whining. If you choose “a way of code”, do something about it. Be full of respect for past generations, because the way you talk about old code today, will be the same way people will be talking about your code years from now. Be full of care about future generations, they will have to live with your legacy code. Be full of childish curiosity, embrace failures, be open, experiment and share. Because as I heard recently “sharing is caring”, not only about others, but also caring about yourself. Being proud of what you are doing is most rewarding thing in our life. Be proud of who you are, don’t accept mediocrity but don’t punish people for it. Maybe, just maybe they didn’t choose “a way of code” and that’s fine. That’s good. Don’t make people feel bad about (I know it is hard, trust me).

So if code is bad, change it, if your not happy with a conference talk, submit your own, if you think your manager is an ass hole, become one, for a moment, to see it , to feel it. Do anything, to be proud of who you are, to be proud of your work.


Your inbox and where we can traverse from there

0
0

I love when my brain has some spare cycles. I love this feeling of being bored. And I love this moment when one of my brain’s cells wake up and shouts out loud, “hey, let’s do something”. In these moments everything looks like a fun thing to do, the next most important project that is going to change the world. I have so many such projects, not finished, never actually fully implemented, the next big thing. Because life is too short to finish projects:).

This time, one of my projects turned into something that works, and I have been using it for the last two years more than 20 times, a Neo4j workshop.

It was a dark, snowy night in Cracow, Poland when said to myself, “hey, what if, I put all my emails into Neo4j”. Doesn’t this sound like a fun project? Of course, it does. You may ask, so what’s the business value of such project. Let me explain you, my business value minded people, fun is the ultimate value, together with exploration and finding people I can go for a drink. That’s what I call business value.

So let me welcome you to a journey on how to find your “true” friends by analysing your inbox in a graph.

This post assumes, at least, basic knowledge about Neo4j and Cypher. It is for people who already started their exploration of this amazing land. This is just one of many ideas you can play with, to get better feel of graphs.

In case you are not familiar with Neo4j, a my favourite graph database, it should be fine as well, as long as you follow my instructions, have downloaded neo4j copy from http://neo4j.com/download/ and read tutorial how to run neo4j shell :).

Enjoy the ride!

Feed your graph

First of all we will need a way to import all of your emails into graph. This is kind of boring task. So I will not go into much details about implementation, as you can find it at http://bitbucket.org/kcrimson/recograph.

Let’s feed the monster

hg clone http://bitbucket.org/kcrimson/recograph
cd recograph
mvn package
cd target/appassembler/bin
./import-gmail -u <your@user.name> -password <your.password> -f Inbox

`-f` option allows you to set folders you want to fetch. It also supports sub folders, so you can pass for example `-f “[Gmail]/AnotherFolder”`. In order to import all of your emails you can just fetch single folder, called “[Gmail]/Wszystkie”, if you use GMail with Polish language:).

WARNING

GMail users need to enable IMAP protocol.

It will take a while. It will probably fail few times. But don’t worry you can re-run it as many times as you wish, as it will import every email just once, based of message identifier. You can also check file `import-gmail.log` to check for status of the import and lookup for errors. If you have enough patience, after an hour or so (depending on the size of your inbox) you will end-up with `mails` directory with Neo4j graph in it. With a structure which looks more or less like this.

asciidoctor-diagram-classes

In Reply To

So what’s in it? All your mails as nodes with label Message and properties `Subject`, `Message-Id` and `In-Reply-To` and all of your recipients as nodes with label InternetAddress and properties `address` and `personal`. You also find relationships of types FROM,TO,CC and BCC, between your Messages and InternetAddreesses.

This is a simple graph, which doesn’t tell you much about your social life, let’s tune it a little bit.

First thing that came to my mind was, emails threads, how can I find all emails threads in my inbox. After few “I am feeling lucky” searches I found that there are actually two headers in emails, which I can use to explore threads. They are called `Message-Id` and `In-Reply-To`. So it was a  time to materialize few relationships.

CREATE INDEX ON :Message(`Message-Id`);
CREATE INDEX ON :Message(`In-Reply-To`);
MATCH (n:Message)
MATCH (m:Message)
WHERE n.`Message-Id`=m.`In-Reply-To`
CREATE n<-[:IN_REPLY_TO]-m;

What happened here is, first I created two indexes, just to spice up things, sorry, speed up, when I will be marching through the whole graph, to find nodes (mails) which are related. Next is a lovely Cypher query which matches nodes where Message-Id equals In-Reply-To and creates a new relationship. This is the thing I love in graphs, exploring, building layers of knowledge from simple facts, layer after layer, till the moment self-consciousness of graph awakens:)
diag-a8ec4dd1581c8473173459529fcc433f

The longest thread

So as you can see it is no longer a simple graph. We have enriched it with `IN_REPLY_TO` relationship, which points to mail which was a response to another mail.

Now, let’s look for a longest email thread in our mailbox.

MATCH thread=((firstMail:Message)<-[:IN_REPLY_TO*]-(lastMail:Message))
WHERE NOT ()<-[:IN_REPLY_TO]-firstMail
AND NOT (lastMail)<-[:IN_REPLY_TO]-()
WITH firstMail, length(nodes(thread)) AS threadLength
RETURN firstMail.Subject, threadLength
ORDER BY threadLength
DESC LIMIT 5;

Isn’t it lovely? I will print this query, someday, on my t-shirt. It is so beautiful.

To be honest it took me some time to make this query use the power of Cyhper, in its pure form. The trick is in the question I asked myself. What is email thread? For the sake of simplicity, something that has beginning and end. I know it is wrong, because email threads are actually trees, not lists, but this is something for another post, to explore. I had to find first and last email in the thread. Once you draw it on a piece of paper it is easy. First email in thread is the mail which doesn’t have outgoing `IN_REPLY_TO` relationships, NOT ()<-[:IN_REPLY_TO]-firstMail, and last email is the email which doesn’t have incoming `IN_REPLY_TO` relationship, NOT (lastMail)<-[:IN_REPLY_TO]-(). Just few ASCII arrows away.

Who is my friend

This is a place when things get even more interesting. Once I have email threads I can try to explore clusters of friends. But let’s come up with definition of friend. If I have your email address, it means I am your friend (it is not so straightforward in Poland :)). It leads to interesting observation, and another layer of facts. If I send email to Tomasz and Kuba, it means they are my friends. But when Kuba responds to all, adding Wiktor, it means he knows Wiktor, but Wiktor not necessarily knows Tomasz and me. And here is another crazy query, with even more arrows.

OPTIONAL MATCH (m:Message)<-[:IN_REPLY_TO]-(n:Message)-[:TO|CC|BCC]->r
WHERE NOT m-[:TO|CC|BCC]->r
WITH n,r
MATCH n-[:FROM]->f
MERGE r-[:KNOWS]-f;

So I am looking for all recipients of the email, and check if there are relationships to these recipients in a previous email in the thread. For these recipients I create `KNOWS` relationship.

diag-07280ed1adcf5d0de72cc59fe65e654d

Where did my cluster go

And last but not least. Clusters of friends. This time we will use well-known algorithm, called local clustering coefficient. I am not going to bore you with details, but in short, it counts number of relationships between your direct friends versus number of potential relationships between them.  Which in short, looks like this

MATCH (a)-[:KNOWS]-(b)
WITH a, count(distinct b) as neighbours
MATCH (a)-[:KNOWS]-()-[r:KNOWS]-()-[:KNOWS]-(a)
WHERE exists(a.name)
WITH a, neighbours, count(distinct r) AS connected_neighbours
WHERE neighbours>1
RETURN a.name, (2*toFloat(connected_neighbours))/(neighbours*(neighbours-1))

In short clustering coefficient equal 1, means that friends of this person know each other very well, really close circle of friends. We could also explore complex world of cliques, for example Bron–Kerbosch algorithm. One interesting fact, finding maximal clique in a graph is on the list of the most complex computational problems, as listed on Wikipedia.

And that’s all folks, I hope you enjoyed it, especially the moments when out of simple facts, we were able to build knowledge.

There is of course a lots of things we can, like adding time dimension to our graph. We can use `Date` header and spread time tree and look at who responded to our emails within minutes or days:).

I hope also I have shown you that Cypher is a great language, but as with every abstraction you have to unlearn old tricks.

Happy traversing.



Frameworks, libraries, languages and deconstructing bullshit

0
0

Actually, when I was writing this article I was on vacation, relaxed, far away from work, with limited access to Internet, reading books, enjoying time with kids. It is just an example of what happens when I spend too much time away from coding. I start to think. Seriously. I recommend this exercise to everybody.

Oh, and one more note, this article contains strong language. Some statistics, word bullshit is used nine times (including this sentence) and word fuck is used two times (including this sentence). You are likely to disagree with my points, and that’s ok. It is ok to disagree and have different point of view, and even call me dump. I would love to hear some arguments with this article, because it would mean I am not surrounded by zombies. This would bring some hope into my life.

I’m tired. This is the feeling I’m hiding deep inside of me. It is not a burnout, because I have it under control.

But I am tired. After 20 years (or more, depends how you count), I can say it out loud. I am tired.

There have recently been a couple of blog posts, articles on happiness level in our overpaid, teased by recruiters with shining like a diamond benefits, eco offices with best of breed fair trade coffee, thin and lightweight Macs, scrum owners (actually psychologists in disguise, and you know what I think about psychology) who are here for you to listen and tokenize and parse your feedback, with tons of empathy for your everyday struggle with The System.

The System, the Monster your created, so later on you can refactor it, using new shining Sword Of New Technology Blessed On Another Cool Conference By Some Unknown Priest of Whatever. It reminds me about the “DragonHeart” movie, where the last living dragon, played by Sean Connery, makes a deal with dragonslayer. The dragon rides villages, burns and destroys everything, then comes the dragonslayer, he “slays” the dragon in well played “heroic fight”, gets the prize and they move to the next village. The System is the dragon and we are the dragonslayers. We get prizes, and move on. The Dragon (khmmm… The System), stays the same. Complex, with dependencies descriptors, getting bigger and bigger, and soon, all our microservices will turn into The System. Where information density is equal on all levels of abstractions, where partial decomposition is impossible and database (this time NoSQL database) is central design lock.

Bullshit.

Don’t get me wrong. I am part of all of this. It is not like I am complaining because I am not part of this craze. I got angry  when somebody wrote an article in Polish blogosphere about getting government to regulate this market as it creates huge gap between people and it’s just unfair, destabilizing economy and other interesting observations.

People are asking me what I do for a living. They are like, really? Can I be a programmer too? Can you teach me this stuff? They give up quickly, because what motivates them is money.

What motivates us? Mastery (and money, you hypocritical bastards). Getting better at things. We have even bring back to life a centuries old idea of craftsmanship.

You say, it is so cool we can learn, improve, gain experience, continuously. Some say, it is part of our work ethic (and even write books about it). And I say bullshit, bingo. Double soy latte, bullshit. Who gave me the right to say so? Let me tell you my young friend. I gave myself such right, right to be honest with all of you, and yes I was not touching lemon vodka since last conference after party🙂. I am relaxed as hell.

I am feeding my brain with whitepapers and paperswelove since 1998. I consume weekly, on average one PDF, sometimes I find monsters like “Software and Hardware Techniques for Efficient Polymorphic Calls”, which took me a month to digest. I still don’t understand last two paragraphs, which makes me feel guilty. Yes, I am talking about this feeling, when you go to bed too early, with a PDF half printed, with links on Twitter you put on your TODO list, with all tweets from your wolfpack, who tweet on lovely, just found paper from 1976 on virtual machines construction, “just in time compilation” and other stuff. You feel guilty for sleeping, while others are expanding their boundaries of understanding. And what do you do? On a next meetup, conference you meet a group of young adepts of dark arcane magic, and you pass the guilt. You tell them about message selectors, single dispatch and overhead of dispatch table row compaction algorithm. They feel guilty, you earn +10 points in necromancy, and you’re free from guilt. Nice trick. This is how it works. Our industry is built on vicious circle of guilt.

Where is the trick? Why can’t we break this cycle? Because our brain is limited, our time is limited and yet it’s evolution, baby. One paper leads to another. One research didn’t prove anything but built foundations for another research. Because we made all so complex, adding layers of abstraction, to simplify things, that we failed (or forgot) to make things simple. Thus, one paper is never enough. Every paper has references section, which is like an address to a next drug dealer. And we need drugs to silence this feeling, this scratching of the soul, that you don’t know enough. It is so hard to admit: oops I didn’t read the original Smalltalk paper; I don’t know that Strongtalk made foundations for inlining in JVM (or it was Self? fuck it). You see. Don’t you feel like shit? I feel like a king. Have you ever thought, if a person with 20 years in business feels like this, what people, new to this whole circus, feel? I am talking about empathy. Yes, your are reading the blog post of a person, who starts his presentations with the sentence “in general, I don’t like people”🙂.

We are in an unpleasant place to be, despite salary, job opportunities, free gym, free beer and whatever free HR department comes with. We are under pressure from our peers to understand, not to stop in our efforts. In the meantime, our lives become empty as we lose track of our ultimate goal: to conquer The System, to work less, and have more fun. Life is not a game. It is only a chance you have, so don’t waste it on feeling guilty.

I can rant for chapters about other tricks we are playing with ourselves. Like hiring for culture fit, 20% of time for doing cool stuff at work (if your are not doing cool stuff at work and company needs to give you 20% special time slot, something is wrong). But it is waste of your and my time. And let’s not cheat ourselves. Somebody needs to maintain the shit I created. Life is not all about roses and love, sex and rock’n’roll. Sometimes you need to dive in shit until you smell it.

There are two (or more, I will not count) things which amplify this behaviour, so let’s find the leverage point (oh, lovely systems thinking).

First thing is hard, like really, really hard. It is so hard, I will not even try to tackle it. I gave up a long time ago. But I am talking about  it when my neurons are floating in toxic liquids, with cherry flavour. People who party with me are so tired of this topic, that they usually release another bottle of poison, just to make me stop talking about it. We don’t have foundations, like 10 papers which define software engineering, which define our discipline, which, if we follow them, The System, the dragon will be under control. It will not be killing random people with fire. What’s funny, some of us say, we have. We have our 10 commandments. But it is like with the religion. Every camp has different rules and as usually with us, humans, pitiful, war loving animals, we fight each other. High Priests of Higher Kinded Types camp fight with knights from Order of Temple Of Object Oriented, wearing steel and composite helmets Paladins Of Strongly Typed Church Of Truly One Type To Rule Them All are filling battle fields with bodies of Dynamically Typed Ninjas of Duck Typing. Somewhere behind all of this, Monks Of Curly Braces train new armies to, someday, confront Reactive Functional Hippies Of Evergreens. And, each camp has its prophet, and priests, and followers. They invest their time in finding new souls to join their ranks. This is how it is. One giant battlefield, and every project is a battle and every line of code is a prove. That this is bullshit. And yet, don’t forget about machine loving Transformers Of Mechanical Sympathy, kicking asses of Viking Warriors Of Immutable Containers. Everybody fights for your soul, yours, you young padawan. Will we find peace? Maybe, someday, but I am not brave enough to follow this path. Because, I am tired.

So what’s the second thing? This is something we can try to change. Together, create an utopia, where not knowing is cool, where new people will feel welcomed. But there always will be people who know and want to share their path, their experience and time.

Hey, wait, aren’t we doing it already? Like sharing on conferences and meetups?

Yes, but I think, it is just a play. Conferences have their goals and targets. They usually are part of one or another camp. We have conferences for A and B, and talking about C ,and where D is the new cool. That’s the first challenge.

The second challenge is:  “celebrities”, “hot names”, “he always has room full of people”. Stars. I am not saying these people are doing a collateral damage. No, not in general (but some of them should get life time ban of public speaking, it’s just my personal opinion). Yeah, I am like shitting in my own nest (old polish idiom). But, you know what? It’s all bullshit. Being nice for the sake of being nice, isn’t part part of my work (or life) ethic. Deal with it and I will deal with consequences.

We are like masonry. It’s hard to join our secret club, it is hard to get to a new level. Because, we work together, review our presentations together, travel, and party together. I hear it more often these days. That we even create our own ghettos during parties. There are special speakers dinners, and speakers rooms, and speakers pub crawling, and special insight code, language we use.

Some people even make a living out of this, not having written production line of code in ages. What a bullshit.

I have recently met two of high rank speakers and tried to talk with them. They were so focused on defending their way of seeing things, that I felt like shit after the discussion. Then it struck me. They were protecting their job security. It was more important for them to protect their dragon, then to be open, honest, ready for discussion about framework, technology or language they promote. If you don’t have enough patience, if you think speaking at main stage makes you a superior human, you are in a wrong business.You write shitty code too (or don’t write it at all, bingo), you feed the dragon like we do. What makes you special?

Don’t let me continue. Yes, I know it is extra work, extra effort. But we are in this together. Together, we shape this industry. Without the audience, we are nothing. Our ideas are dead, without people who actually implement them. They have important feedback for us, and this feedback is: stop creating ghettos. This is my another observation. People are either afraid to come and ask, or we are building walls around us. I thought, hey, I am not nice, it is not welcoming when you say “in general, I don’t like people”.

Recently, on one of the conferences, I understood, it is not a problem. It is just a filter for people who don’t get sarcasm. The problem is, even when people get sarcasm, they are trapped in this guilty hell, because they follow us on Twitter, they see my sick tweets at 2:05 about another PDF I found, and they didn’t read it.

Hey, I say it is ok. It is cool not to know. I have read it. Stay cool, give me a shot of something strong and cold, and I will tell you what I understood, why I think it is important.

Somebody asked me, why I do it for free. I don’t know. Maybe, I like when I turn my sleepless nights into something worth it. And, of course, I love to be the wisest man on the block:), c’mon, deep inside it is all about this🙂. We are like a 10 year old boys playing football in the backyard, and somebody just showed a new trick.

I’m tired of chasing my dream alone. Having to read all these papers. Not knowing what’s next.

Tired of our wars, because they are pointless, no matter FP, OOP, AOP, FRP, at the end there is CPU and OS. And they don’t care. And we created abstractions which are here rather to make people feel stupid, than to make things simpler. We feed new dragons, when the old one is still alive and doing well.

I am dreaming about a place where there is no audience and speakers, where there is one afterparty, where lack of knowledge is privilege, and if you know and understand, it is your duty to share it. Where there are no camps, no knights, no dragons. A sacred place where, before you enter, you have to leave all of your weapons, langs, frameworks and libs, at the front door. Where the only stars are inside us, lonely stars looking for understanding to become one.To explore space. To enjoy the things we don’t know we don’t  know.

I am naive.

I know, this is my utopia. I need to free myself from being tired talking about technology, about frameworks, and libs, and containers.

I am not saying they are not important. I see them like a forest full of trees. They hide the truth, that  the king is naked and the princess is ugly, and there is no handsome prince on a white horse.

The truth is, deep inside, behind all of the abstraction layers, there is OS and CPU, deep inside, behind all of ORM frameworks there is SQL and deep inside, behind all of MVC frameworks and JS libs dependency hell there is DOM and HTTP, for God’s sake.

I would love to see more and more talks about foundations, about things that build frameworks, languages and libraries, layers of abstraction below. These are stable layers of the system to which you should design dependencies of your knowledge and experience. Build on things that will not change in a year, oops, month.

Would you do me a favour? Next time you are going to talk about a new framework, rather talk about the layer this framework tries to hide and “simplify”.

… yeah, this article was not reviewed, accepted or read by any of my friends and peers, because I don’t strive for perfection, I love chaos and improvisations. So sorry for all mistakes🙂

 


Alive and hacking : revised

0
0

Some time ago I found my old articles, from previous blog. Some of them are 5 or more years old, written in Polish, from times where I was spending more time far away from coding. In those times, I was focused on managing and “architecting”. It gave me a strange perspective on things. Few months back, I woke up with this idea to comment and criticize my own thoughts. I don’t know if you are going to like it, but at least for one person, it was fun 🙂 And it was me.

Before we start, lots of love and thanks to my wife, who took this burden on her shoulders, and translated these articles.

I include the whole article with my comments and thoughts inlined. Enjoy, or not 🙂

The 1980’s. Simple Minds and their classic Alive and Kicking (especially in
live version).

I have not been here for a long time. I neglected this small group of
readers, the blogosphere, and Googlebot. In the meantime, I changed my job, explored Northern Europe, survived the deadly “afterparty” after the first edition of the refreshed Confitura, and started a PhD in JBoss 4.2.3 GA.

I shut myself in my small space, at the new desk with a view of Krakow and repeat like  mantra the mvn clean package, run and subsequent curse through gritted teeth and git reset. I turned off Google Reader, InfoQ and a few other “disturbers”. The only entertainment is to keep alive the noble idea of “zero inbox”.

I think I owe you a little bit of explanation. Back in days, when I was writing the original version, I worked as a “chief architect” (I still don’t know what that means). I worked with a system with ~500 kloc of Java + ~500 kloc of Coldfusion and around ~400 kloc of JavaScript (jQuery) code. It was like a baby born after a heavy drinking party with T Rex, xenomporh and Princess Xena, but you would not know who had sex with who.

Everything was dipped like nachos in spicy salsa in JBoss 4.2.3.custom_fixes_and_extensions, something I consider last stable version of JBoss. This application was a pure proof that it is not a challenge to make simple things simple, but it is a true mastery to turn CRUD application into “feature set” orgy, customizations “gang bang” and “we have wizards everywhere” extravaganza. Deal with it.

For clarity, the code I am working on, is not bad, terrible, unreadable or fatal. It is also not that it underwhelmed me, disappointed or plunged into the black as espresso despair. The man already knows what to expect, and accepts the fact that the client is the Lord and basta. And a racial purity of the code, its kosher, remains in a beer reverie in a cigarette smoke.

Actually, it was. Bad, terrible, unreadable and fatal code. This is, my dear friend, the first stage of working with legacy systems. Just admit it and say it out loud: “this code is garbage”. But do it one time. I think we spent too much time on whingeing: “who wrote this code?”, “this is shit”, “I am not going to work with it”, “this is pure spaghetti with hand grenade in it”. It is, and so what?

Keep in mind, you Mr. Perfect High And Mighty, that probably somebody, in 10 years from now, will tell all of these things about your “shiny code”.

Yes, somebody had a bad day 20 years ago. Somebody was forced to follow “architecture guidelines”. Somebody didn’t care at all, because he knew he would be moved to another project in a week. I can think about hundreds of reasons why this code is bad. There are days in my life where I shouldn’t be allowed to code or even get close to an IDE. Because I am fed up, because I am tired or with a mountain high hangover. Or my kids are sick, or my parents are not well.

First of all, you and I, we are human beings and work is not life. (Maybe it is when you are 20 or 30 but not when you are 40 🙂 )

So just don’t try to improve your self-esteem by saying: “I would never ever write this code this way, these people were stupid”. I bet you would have written even worse code in the same context this person was in.

So, “yes, this code is bad”, end of story. Let’s do something about it. The first step in a whole organization is to admit how bad it is and embrace the fact that it actually makes money, because you still have your job. Period.

So what pushes me to these late evening thoughts? For some time now, I live outside the mainstream of a TDD, BDD, DDD and Software Craftsmanship (now these toys do not quite fit into my sandbox). I realized how hard it is to work without these tools. How hard, at times, it was when I could use them on a daily basis but it is even harder to survive without them (maybe it’s just a matter of habit). Currently, with bare hands, I decompose the code structure, make destruction of local dependencies, discourage next application layers, and remove/disassemble piles of false assumptions and recognized truths. My human nature does not allow me, however, to accept the existing state of things and trying to understand the motivations and context.

What did go wrong? Because it went wrong all the way down to the gates of hell. It exploded in our hands right into faces of surprised CEO and other C-level executives. Oh God, this was so much fun talking with these people during these days.

I thought that quality was important. I was so fooled by all this software quality talks that it narrowed my view on the system. Don’t get me wrong, it is important but not when you have to dive into savage. When you are right before jump into shit and you don’t know how deep it is, last thing you need is quality. You need a world-class life support system.

You need a working continuous integration. You need tests which are not “false positive” all the time. You need bloody fast build times so you can experiment with “workarounds” all the day and all of the night. You need static code analysis quality gates which will make sure you will not get a system into worse state with your refactorings. You need  access to production, without stupid procedures which give you systems logs a week after crash. As you are diving into shit, so how will these procedures make your journey more enjoyable?

And last but not least. You have to remove all signs of bullshit in management of your organization, at all levels. All this “our best people are working on it” (of course they are), “we need to break silos” (of course we need to, first start with all these management levels), “we have certain values” (no shit, really?  I am proud of you. Do your values reflect quality of code you have?), “business value is priority” (how about all the money flushed down the pipes with all this technical debt you have?).

Back to the topic.

Fast turnaround and feedback is key. Especially in first days, weeks and months. It means it is more important to have “one click deployment” and  working development environment on people’s laptops than “clean code”. Remember, the deeper you dive, your focus will need to change. But in first moments, everything you do, should be driven by this question: “Will I break the system? Yes? How fast will I recover?”

And this is something I have ignored. I thought that if I had a better coverage, response for class or cohesion metrics, everything would be fine.

It wasn’t.

I try not to search the guilty (git log will not change anything). I assume the good will of creators, unclear requirements and pressure of time.

So where does the reason lie? In JBoss AS. Do not take it too personally. It might as well be the Spring Framework, OSGi, Grails or homemade, tailor-made platform. Each platform/framework, or other kind of animal, carries the initial technical debt. You think that JBoss does not have a technical debt, and its code is clean as the proverbial glass of “wyborowa”. Stop delude yourselves. Every piece of code that was created is burdened with lesser or greater debt.

There is also another category of debt that comes along with a choice of “this only solution”.

Our lack of complete knowledge of the selected technology, gaps in documentation, outdated tutorials, things that developers do not want to speak loud and “miraculous” solutions available in the network. So this is what Dan North presented in the “Deliberate Discovery”. Our debt is what we do not know that we do not know :). And in this way, the simple things, as provided in platform are entwined by “hacks”, aspects, modified libraries just to make ends meet and release it.

I , kind of, still agree with these statements. With a small addition. Very often, we invest in certain technology, make it a backbone of the whole system. We love it so much that we see every problem through it. Even when deep in the heart we know it doesn’t fit our stack, our needs or our infrastructure. But because we have invested in it so much, so many hours, we think it is “the solution” to all of our problems. Sooner or later we are going to create “extensions”, “patches” and “proxies” to justify its existence in our environment. The truth is, when it doesn’t fit, it doesn’t fit. I see this kind of behaviour as one of key technical debt sources. Together with “I know shit about this NoSQL database, but will use it everywhere” and “cool kids use it these days, it has to be good”.

One thing that can be helpful in these moments is architecture decisions log. A place where you store all your stupid decisions with a context. Why have you decided to use X? What kind of assumptions did you make about X? When these assumptions appear to be false, you will have a clear evidence why it is a stupid choice.

And I didn’t have architecture decisions log.

Of course, you can still throw a pebble of our ignorance in the area of the domain business but at this stage I’m not going to wear a noose around my neck.

Now we get to the root of the problem. How important is the choice of our target, unique and beloved platform? In my opinion, it does not quite matter, and regardless of the choice, we incur a technical debt prior to writing the first line of code. And we will amaze future generations by our ignorance.

Oh, how wrong I was.

Yes, it does matter. The technology you choose, will drive your success or failure, will be there with you till the end of your life, will be your “hand of doom” in these lovely moments where whole your SRE team will sit by the camp fire and watch your DC burn in flames.

So why is the technology important? That’s tricky. In my old brain there is this hidden thought, hidden deep in darkest corners of my being.

“It is important to know how easy it will be to remove this technology from the stack, when the time comes”

Really.

There are frameworks, libraries and other stuff, which require you to bond your code with it really, really strong. You have to put import, annotation, extend, here and there. And slowly, your code is strangled into the framework, you cannot move, you cannot breathe, you cannot do anything which is not permitted by “uber framework”.

What if we could build our applications without single import from a framework whose name I am not going to reveal? What if we could build whole systems just using language features and its standard library?

That’s nothing new. Ports & adapters or onion architecture.

But hey, sooner or later, you will have to change your technology stack. Why do you need to change your so-called “business logic” code? It’s orthogonal concern for me.

Don’t let your framework strangle your “business logic” code.

Our debt will grow with each new version of JBoss or Spring to which we will not have time to move our application. Theoretically, the ignorance of the creators of those platforms will diminish with each new version. So lagging is not in our interest. But who can afford it? It is better to run up a debt.

It will not be shown in the annual financial statements anyway.

What’s interesting, we have not yet worked out a clear and effective approach to such problems (talking about “legacy code, architecture and design”). I know you will bombard me now with comments about my reading ignorance. But of course, I did read the “Working with legacy code” and “Refactoring to patterns”. However, each such application with which you will work, is unique in the set “brainfucks” which surrendered its creator. And that is why I believe, that if you seek real challenges, intellectual puzzles, and work that will bring you waves of elation and the avalanche of falls, look for some “legacy” system. Personally, I believe that none of the new, written from scratch applications, in the latest “frameworks”, extremely dynamic languages, giving you sharp tools into your hands, does not make so much pleasure and pride in a job well done as solid “legacy” put in your hands for straightening. Why, you ask?

Welcome to the moment when you start to accuse about lack of diversity and empathy in my way of thinking. But I have stopped worrying about what people thought ages ago (except carefully chosen a small circle of people who I respect).

Ready?

“Everyone can code”. That’s a terrifying idea. We should, of course, create an open, welcoming place where everyone would feel safe, regardless of our background, skills and things that make as different (in the first blink of an eye). The thing that we have to even discuss it, it’s a pity and shame for us as human beings. That we are so miserable, that we put people in boxes, and create boundaries. As we now talk about it, it creates the other extreme of this problem. However, let’s leave it for another discussion.

Do we really believe that the only way to solve shortage of talent in IT is to have more people coding. Really? If it is true, it means that we have also failed as an industry in building right tools, paradigms and practices. Do you remember this joke about pregnant woman and project management? This is what I am talking about.

I have recently read a nice write up about Alan’s Kay idea of Moore’s law in software development. That’s sad, that it didn’t happen. That hardware outperforms software at so many levels. That we are where we are mainly because of advances in hardware. Maybe I am wrong. I hope so.

Good app is built around hacks. This is not TDD, BDD, DDD and Software Craftsmanship but HFDD (Hack it and Fuck it Driven Development). The new better paradigm 🙂

The future of IT, to the fear and general scandal, is programming archeology. And yet it is because of archaeology we learn about the past generations and their customs, we reconstruct course of events, we expand our experience and thus we become better. Sometimes also, we discover an extinct technology, and when we see that it is good, we put the new sticker on it and sell as a breakthrough discovery.

That’s the end. No final word.No closing comment. Just many thanks to Wiktor, Krzysiek, Kuba, Paweł and Jarek. You didn’t know, but you were inspiration too many thoughts in this post. I have a feeling that lots of these thoughts deserve their own post but I am too lazy to continue;)






Latest Images