On teh way

Elasticsearch MySQL Sync Challenge (4): Quality Attributes Tony was called into the leader’s office. After the basic functionality is fulfilled, Tony has started to review the quality attributes of the project. Extensibility “The syncer you have completed works great and it has good extensibility for its relative loose restriction between different modules (input, filter, output). Now, we need one more output choice: to output to MySQL server. How long do you think to finish this feature?” “Em, at least three days, for both coding and testing.” “Fine, go for it.” The process to extend one more output choice is relative simple: Add MySQL output config config package, which is used to mapping config item in yaml to class; Add MySQL output channel, which is the abstraction remote destination; Add SqlMapper , which convert the SyncData to sql statement when SyncData reach the MySQL output channel; The first item is implemented very fast but the seco...

Java JIT Inline This is a follow-up blog that I write after answering a related question on SO . Through this question, we can understand why we need to know the internal of machine/os/language etc. Introduction of JIT The JIT (just-in-time) compiler is a important component of JVM which is used to optimize the performance of Java. As it name indicates, JIT can perform just-in-time compilation, which is the compilation process from byte code into native code of specific platform to make program run faster. And JIT compiler can not only makes standard compilation, but also it can performs optimizations like OSR (on stack replacement) , class hierarchy analysis etc. Today, we will see how JIT do inline optimization and how it affects the performance of program. Inline Function First, we get to understand what is inline function . To inline a function is to substitute the function call with the function body. Compiler can inline some small hot functions into cal...

Spring Boot: Customize Environment Environment variable is a very commonly used feature in daily programming: used in init script used in startup configuration used by logging etc In Spring Boot, all environment variables are a part of properties in Spring context and managed by Environment abstraction. Because Spring Boot can handle the parse of configuration files, when we want to implement a project which uses yml file as a separate config file, we choose the Spring Boot. The following is the problems we met when we implementing the parse of yml file and it is recorded for future reader. Bind to Class Property values can be injected directly into your beans using the @Value annotation, accessed via Spring’s Environment abstraction or bound to structured objects via @ConfigurationProperties. As the document says, there exists three ways to access properties in *.properties or *.yml : @Value : access single value Environment : can access multi...

Elasticsearch Endless Queue? This day, Tony was notified that the Kibana which is used to query logs can’t working now. This a crucial part for log aggregation in new log system used in Microservice environment , so Tony set out to deal the it. Rejected Task Found nothing related in the Kibana’s log, Tony decided to checkout the Elasticsearch’s log. He found that there exists many long EsRejectedExecutionException . [DEBUG][o.e.a.s.TransportSearchAction] [xLI0MAD] All shards failed for phase: [query] org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: rejected execution of org.elasticsearch.transport.TransportService$7@30fadff6 on EsThreadPoolExecutor[search, queue capacity = 1000, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@6c9b1b8d [Running, pool size = 25, active threads = 25, queued tasks = 1921, completed tasks = 169477]] Just google the exception show some SO threads which introduced the thread pool and request q...