On teh way

This week, we are going to talk about a problem we may not notice in common concurrency code but may bite you later. Synchronize on Method Call Quick Question See the following code: thread one will start first, then thread two. So, will the first element of list printed? Or deeper, can second thread enter that synchronized block? List list; // thread one synchronized (list) { while ( true ) { try { Thread.sleep( 1000 ); } catch (InterruptedException e) { e.printStackTrace(); } } } // thread two synchronized (list.get( 0 )) { System.out.println(list.get( 0 )); } Answer The answer is yes. Do you get it? let’s analyse what the essence of this problem. First, we can simply find the the lock of list is always hold by thread one for its infinite loop. So, whether the second thread can enter that synchronized block depends on whether we need to get the lock of list. Now, we need to understand: whether we get option one or option two wh

Basic wait When we want to coordinate multiple threads to run in order, there are many useful synchronizer classes in Java library for us to use, like Semaphore , CyclicBarrier , CountDownLatch etc. A common usage of synchronizer class is consumer and producer problem. If we decide to use some basic method from internal lock, we can do it like following: // thread one while (ready) { obj.wait(); } // thread two ready = true ; obj.notify(); // or, depends on situation, obj.notifyAll(); Or we can use explicit lock: Lock & Condition to do similar thing. condition .await () ; condition .signal () ; And the above code is more a practice than a real usage. If you really need to write a producer and consumer solution, BlockingQueue may be more convenient to do it. waitForAny When we get familiar with the common use of wait, we are going to implement waitForAny utility function based on other library code of java. Definition As what we always do, we

I always use some new release of Linux distribution, so some software froze from time to time. In order to restart it, kill command can be very handy to use for it so fast and saves lot of time compared with UI prompt. So today, we learn how to kill effectively. Kill First, we should understand how to use kill . But we are surprised to find the man kill only tell us the common format: $ kill [signal or option] PID (s) But what’s the signal? Which one should I choose? Signal is a kind of inter-process communication which we can take it as a message sent between processes. And we usually use the following three signals: Signal Name Signal Value Behaviour SIGHUP 1 Hangup SIGKILL 9 Kill Signal SIGTERM 15 Terminate Some kinds of signal can be ignored, like SIGTERM ; but some can’t, like SIGKILL . So what we commonly use is the following: kill - 9 pid Find the PID

In this blog, we will learn some basic concepts and basic API about Hadoop. Overview of Hadoop Hadoop is an Apache open source framework written in java that allows distributed processing of ‘Big Data’ across clusters of computers using simple programming models. What Is Big Data? Hadoop is the software to handle big data, but what is big data? Big Data has following features: Big volume Big velocity Big variety Structure of Hadoop The following is some components of Hadoop: A distributed file system called Hadoop Distributed File System (HDFS) to store data A framework and API for building and running MapReduce jobs to handle data Hadoop common: utilities Hadoop YARN: job scheduling and cluster resource management Range and Aim In order to achieve better effect of learning, we should have a clear aim and detailed road map. So we first define the range we will learning: basic concepts and architecture of hadoop basic API Aim: understand

Today, we implement a sorted linked-list and try to feel how to write beautiful code. Question Implement a sorted linked-list with max possible value known and following declaration: Class SortedList { public SortedList ( int maxValue) { //... } public void insert ( int i) { } } Simple Version What comes to my mind, at the beginning, is some code like the following: public SortedList0 ( int max) { } public void insert ( int i) { if (head == null ) { head = new ListNode(i, null ); } else { ListNode prev = null , next = head; while (next != null && next.getVal() < i) { prev = next; next = next.getNext(); } if (prev == null ) { head = new ListNode(i, head); } else { prev.setNext( new ListNode(i, next)); } } } This version works, but we can see there are some special cases need to handle which make o

In this thread, we come to the issue of concurrent access of LevelDB. As a database, it can be concurrently accessed by users. But, it wouldn’t be easy to provide high throughput under product load. What effort does LevelDB make to achieve this goal both in design and implementation? Goal of Design From this github issue , we can see LevelDB is designed for not allowing multi-process access. this (supporting multiple processes) doesn’t seem like a good feature for LevelDB to implement. They believe let multiple process running would be impossible to share memory/buffer/cache, which may affect the performance of LevelDB. In the case of multiple read-only readers without altering the code base, you could simply copy the file for each reader. Yes, it will be inefficient (though not on file systems that dedupe data), but then again, so would having multiple leveldb processes running as they wouldn’t be able to share their memory/buffer/etc. They achieve it by adding a l

Before we started, we have to review the definition of nested class: Nested classes are divided into two categories: static and non-static. Nested classes that are declared static are called static nested classes. Non-static nested classes are called inner classes. Features Nested class is very common in Java code, and provides some benefits as oracle document says: It is a way of logically grouping classes that are only used in one place; It increases encapsulation; It can lead to more readable and maintainable code. And here, we are going to focus on the features and convenience that nested class bring to us. The following listed items are some of them: They can access each other’s private members directly Inner class can access outer class’s field directly Secret of Access Private Member Now, we move to a simple example of static nested class to see how Java make it. As the comment in the code pointed out, nested class and outer class reference

Introduction nicstat is to network interfaces as “iostat” is to disks, or “prstat” is to processes. It is designed as a much better version of “netstat -i”. Its differences include: Reports bytes in & out as well as packets. Normalizes these values to per-second rates. Reports on all interfaces (while iterating) Reports Utilization (rough calculation as of now) Reports Saturation (also rough) Prefixes statistics with the current time With the help from nicstat , we can identify whether distributed Java application is saturating the network by view the utilization percentage of specific interface. Download Sourceforge Download Link Build Following the guide of README.txt, we build like following: $ mv Makefile.Linux Makefile $ make mv nicstat `./nicstat.sh --bin-name` Install $ make install gcc -O3 -m32 nicstat.c -o nicstat In file included from /usr/include/features.h: 392 : 0 , from /usr/include/stdio.h: 27 , from nicsta

public class DefaultConstructor { private static class B { } public static void main (String[] args) throws IllegalAccessException, InstantiationException { B.class.newInstance(); } } When I use reflection to create an object from B, it give me the following exception: java.lang.IllegalAccessException: Class DefaultConstructor can not access a member of class DefaultConstructor$B with modifiers “private” Let us see which member the DefaultConstructor is trying to access by viewing the source. Source Code The following is part of source code of Class#newInstance() Constructor<T> tmpConstructor = cachedConstructor; // Security check (same as in java.lang.reflect.Constructor) int modifiers = tmpConstructor.getModifiers(); if (!Reflection.quickCheckMemberAccess( this , modifiers)) { Class<?> caller = Reflection.getCallerClass(); if (newInstanceCallerCache != caller) { Reflection.ensureMemberAccess(caller,

Origin When debugging some Java applications or learning the internal implementations of some library class, we need to read the source code. So I download source from some sites. Soon, it caused some inconvenience. The class file will update whenever the package updates, now it change to: openjdk version “1.8.0_102” But my source code is still the old version. In the intellij idea, it will remind me that class file and source code is mismatched and has some diff. It may be no harm at first glance, but when we need to find the line where class file throws an exception, we may find an mismatched line. Like above picture shows, we can’t following the line number to find the right line if source code and class file is mismatched. And for the same reason, you can’t set a break point in library class file. Solution In order to sync the source code with class file automatically, we have to install a source package, making them updating at the same time. The following

In the Generic Common Usage , we introduced some common usage about java generic: extends , ? super T , wildcard usage. Now, we come to how generic is implemented and what this implementation brings to us. The implementation of generic – erase We all hear that Java generic is implemented by erase, but what does that mean? More specific, how erase affect the byte code, and Java run time environment. Byte code In order to achieve the back-compatibility, Java use erase to implement generic. This is what we commonly heard, and how erase makes it? It’s simple. Making all byte code same, whether it uses generic or not. In this way, you can invoke old library without generic and they can invoke new generic code without worry. Just because the byte code makes no difference. we can see from following snippet that the byte code use generic is almost the same with code not using generic. // pre Java5 generic code public static void swapAgain (Object[] ts, int i, int j)

This time, we will solve a common problem – matrix transposition. Description and simple solution Give matrix of n*n , return the transposed matrix. We can come up with the simple solution from the definition of matrix transposition: for x < n for y < x swap( a [x,y], a [y,x]) Different problem It’s so easy, right? Now we move another similar problem: how to efficient transpose a very large matrix(n*n) on the tape? They are similar, both want to transpose a matrix. They are also very different for the data source affect how we can retrieve data effectively. When we have to read data from tape, element can’t be randomly accessed any more. Upper solution works, but too slow. int n = 200 ; // I use LinkedList to simulate the tape LinkedList<Integer> matrix; // init code long start = System.nanoTime(); for ( int i = 0 ; i < n; i++) { for ( int j = i + 1 ; j < n; j++) { final int origin = i * n + j; fin

I was asked some confusing point about generic during my interview, so I review the generic after it. Following is some note. Basic use First, we write our simple container: public class Container<T> { private T[] a; private T get ( int i) { return a[i]; } private void set ( int i, T t) { a[i] = t; } } In our example, generic only work like some place holder which can be replaced by Integer or String or Object. bounded type parameter Now, we move to more complex usage. We add one more method to our container: addAll to add all element from other Collection. public void simpleAddAll (Collection<T> c) { for (T t : c) { list.add(t); } } This works, but somewhat limited. We can only add the same type element into our container rather than like what standard library can do. Learning from the jdk source, we can change our method into: public <E extends T> void addAll (Collect

Origin When I optimize a mysql pagination interface , I want ot use Redis cache to store some result to speed it up. In order to estimate how much memory I will spend, I have to do some memory estimation. Solution This solution comes from this stackoverflow question . Steps Write the following source: package memory; import java.lang.instrument.Instrumentation; public class ObjectSizeFetcher { private static Instrumentation instrumentation; public static void premain (String args, Instrumentation inst) { instrumentation = inst; } public static long getObjectSize (Object o) { return instrumentation.getObjectSize(o); } } Add the following line to your MANIFEST.MF : Premain-Class: memory.ObjectSizeFetcher Use getObjectSize: package memory; public class C { private int x; private int y; public static void main (String [] args) { System. out .println( ObjectSizeFetc