發現要堅持寫博客真的是一件很困難的事情,各種原因都會導致顧不上博客。本來打算寫自己動手實現orm,看看時間,還是先實現一個動態sql,下次有時間再補上orm完整的實現吧。 用過mybatis的人,估計對動態sql都不陌生,如果沒有用過,就當看看熱鬧吧。我第一次接觸mysql是在大四的時候,當時就覺得 ...
發現要堅持寫博客真的是一件很困難的事情,各種原因都會導致顧不上博客。本來打算寫自己動手實現orm,看看時間,還是先實現一個動態sql,下次有時間再補上orm完整的實現吧。
用過mybatis的人,估計對動態sql都不陌生,如果沒有用過,就當看看熱鬧吧。我第一次接觸mysql是在大四的時候,當時就覺得動態sql這東西很牛,很靈活,一直想搞明白怎麼實現的,儘管當時已經能夠寫ioc,mvc和簡單的orm框架(仿mybaits但是沒有動態sql部分),但是仍然找不到mybatis核心的動態sql到底在哪實現的,怎麼實現的,可能是那些代碼太繞根本沒法看懂,直到目前,我都沒有勇氣去看mybatis的動態sql部分,大概是天生對演算法有莫名其妙的敬畏吧。
幾年前因為想做一個配置平臺,想用解析型語言替代java的實現,可以讓配置人員在頁面上方便的編寫少量代碼實現複雜的業務邏輯(包括資料庫操作)。當時java已經有js解析引擎,但是大多數人都說效率太低,不知道我發什麼瘋就想到自己實現一個解析語言。不過實現自己的語言也是我一直的夢想,解析語言相對編譯型語言入手簡單,於是我就果斷動手了,寫完才知道,其實自己的實現估計還沒有當時的js引擎效率高,那時的我真的是很年輕很簡單。今天談到的動態sql實現其實就是受到那時候解析語言的啟發。
廢話不多說直接開始聊動態sql,請看下麵例子,先聲明這裡的例子並不是一個正確的sql的寫法,只是想寫一個儘量複雜的嵌套結構,如果把這種複雜的情況實現了,那麼簡單一點的就更加不在話下了。
delete from pl_pagewidget <if test="widgetcodes != null"> where pagewidgetcode in <foreach collection="widgetcodes" item="item" index="index" open="(" separator="," close=")"> <if test="index == 0"> #{item} </if> <foreach collection="bs" item="b" index="index1" open="(" separator="," close=")"> #{b} </foreach> </foreach> </if> <if test="a != null"> and a = #{a} </if>
要實現解析出上面例子的sql,首先一個難點類似是test屬性里的條件怎麼判斷真假,不過這個難點在struts2中學到的ognl表達式面前就比較小兒科了。不知道有麽有朋友遇到過一個比較奇葩的現象,就是有時候明明在mybatis動態sql中寫如下表達式,但是當n=0的時候居然是滿足條件的也就是test里的值是false,0居然不能滿足這個表達式的條件,這裡就是ognl庫的原因了。沒辦法它就是這麼玩的,當成特殊情況記住就可以了
test="n != null and n !=''"
ognl表達式使用很方便如下
import java.util.HashMap; import java.util.Map; import ognl.Ognl; public class OgnlTest { //輸出結果:false public static void main(String[] args) throws Exception { String con1 = "n != null and n != ''"; Map<String,Object> root = new HashMap<>(); root.put("n", 0); System.out.println(Ognl.getValue(con1,root)); } }
要實現解析上面例子的sql,第二個難點就是雖然這個sql披上一層xml的皮就是一個標準的sql,如下
<sql> delete from pl_pagewidget <if test="widgetcodes != null"> where pagewidgetcode in <foreach collection="widgetcodes" item="item" index="index" open="(" separator="," close=")"> <if test="index == 0"> #{item} </if> <foreach collection="bs" item="b" index="index1" open="(" separator="," close=")"> #{b} </foreach> </foreach> </if> <if test="a != null"> and a = #{a} </if> </sql>
但是要解析上面的xml和我們平時不一樣,這個xml是標簽和文本混合的,正常我們開發中應該很少會用到解析這種xml。不過我們常用的解析xml的工具dom4j其實可以很好的解析這種sql,只不過很少可能用到。Element類的content()方法就可以返回一個Node的集合,再通過遍歷這個集合,判斷每個Node的類型就可以了。解決了這兩個重點,只需要加上一點技巧就可以解析這種動態sql了。
我用到的技巧是根據java語法格式得到的啟發。比如java中有局部變數和全局變數,不考慮引用傳遞這種情況,如果全局變數int i = 1;方法裡面傳入這個全局變數,然後在方法裡面修改,在方法裡面看到的是改變後的值,但是在方法外面看到的仍然是1。這個現象其實學過java應該都知道。還有就是當方法調用的時候,方法裡面可以看到全局變數,也可以看到局部變數,方法調用結束後局部變數會被清空釋放(看垃圾搜集器高興)。介紹了這些直接上代碼了
import java.io.StringReader; import java.text.SimpleDateFormat; import java.util.Arrays; import java.util.Date; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.regex.Matcher; import java.util.regex.Pattern; import org.apache.commons.collections.MapUtils; import org.apache.commons.lang.StringUtils; import org.dom4j.Document; import org.dom4j.Element; import org.dom4j.Node; import org.dom4j.Text; import org.dom4j.io.SAXReader; import com.rd.sql.Attrs; import com.rd.sql.BaseNode; import com.rd.sql.NodeFactory; public class SqlParser { private Map<String,Object> currParams = new HashMap<String,Object>(); /** delete from pl_pagewidget <if test="widgetcodes != null"> where pagewidgetcode in <foreach collection="widgetcodes" item="item" index="index" open="(" separator="," close=")"> <if test="index == 0"> #{item} </if> <foreach collection="bs" item="b" index="index1" open="(" separator="," close=")"> #{b} </foreach> </foreach> </if> <if test="a != null"> and a = #{a} </if> */ public static void main(String[] args) throws Exception { Map<String, Object> map = new HashMap<String, Object>(); map.put("widgetcodes", Arrays.asList("1", "2")); map.put("bs", Arrays.asList("3", "4")); map.put("a", 1); SqlParser parser = new SqlParser(); System.out .println(parser.parser("delete from pl_pagewidget\n" + "\t<if test=\"widgetcodes != null\">\n" + "\t\twhere pagewidgetcode in\n" + "\t\t<foreach collection=\"widgetcodes\" item=\"item\" index=\"index\" open=\"(\" separator=\",\" close=\")\">\n" + "\t\t <if test=\"index == 0\">\n" + "\t\t #{item}\n" + "\t\t </if>\n" + "\t\t <foreach collection=\"bs\" item=\"b\" index=\"index1\" open=\"(\" separator=\",\" close=\")\">\n" + "\t\t\t#{b}\n" + "\t\t </foreach>\n" + "\t\t</foreach>\n" + "\t</if>\n" + "\t<if test=\"a != null\">\n" + "\t\tand a = #{a}\n" + "\t</if>\n", map)); System.out.println(parser.getParams()); } public String parser(String xml, Map<String, Object> params) throws Exception { // xml = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"+xml; //給輸入的動態sql套一層xml標簽 xml = "<sql>"+xml+"</sql>"; SAXReader reader = new SAXReader(false); Document document = reader.read(new StringReader(xml)); Element element = document.getRootElement(); Map<String, Object> currParams = new HashMap<String, Object>(); StringBuilder sb = new StringBuilder(); //開始解析 parserElement(element, currParams, params, sb); return sb.toString(); } /** * 使用遞歸解析動態sql * @param ele1 待解析的xml標簽 * @param currParams * @param globalParams * @param sb * @throws Exception */ private void parserElement(Element ele1, Map<String, Object> currParams, Map<String, Object> globalParams, StringBuilder sb) throws Exception { // 解析一個節點,比如解析到了一個if節點,假如test判斷為true這裡就返回true TempVal val = parserOneElement(currParams, globalParams, ele1, sb); //得到解析的這個節點的抽象節點對象 BaseNode node = val.getNode(); /** * 實際上這句之上的語句只是解析了xml的標簽,並沒有解析標簽里的內容,這裡 * 表示要解析內容之前,如果有前置操作做一點前置操作 */ node.pre(currParams, globalParams, ele1, sb); //判斷是否還需要解析節點里的內容,例如if節點test結果為true boolean flag = val.isContinue(); // 得到該節點下的所有子節點的集合,包含普通文本 List<Node> nodes = ele1.content(); if (flag && !nodes.isEmpty()) { /** * 這裡表示要進一步解析節點里的內容了,可以把節點類比成一個方法的外殼 * 裡面的內容類比成方法里的具體語句,開始解析節點的內容之前 * 先創建本節點下的局部參數的容器,最方便當然是map */ Map<String, Object> params = new HashMap<String, Object>(); /** * 把外面傳進來的局部參數,直接放入容器,由於本例中參數都是常用數據類型 * 不會存在引用類型所以,這裡相當於是一個copy,為了不影響外面傳入的對象 * 可以類比方法調用傳入參數的情況 */ params.putAll(currParams); //迴圈所有子節點 for (int i = 0; i < nodes.size();) { Node n = nodes.get(i); //如果節點是普通文本 if (n instanceof Text) { String text = ((Text) n).getStringValue(); if (StringUtils.isNotEmpty(text.trim())) { //處理一下文本,如處理#{xx},直接替換${yy}為真實傳入的值 sb.append(handText(text, params,globalParams)); } i++; } else if (n instanceof Element) { Element e1 = (Element) n; // 遞歸解析xml子元素 parserElement(e1, params, globalParams, sb); // 如果迴圈標誌不為true則解析下一個標簽 // 這裡表示需要重覆解析這個迴圈標簽,則i不變,反之繼續處理下一個元素 boolean while_flag = MapUtils.getBoolean(params, Attrs.WHILE_FLAG, false); if (!while_flag || !NodeFactory.isWhile(n.getName()) || e1.attributeValue(Attrs.INDEX) == null || !e1.attributeValue(Attrs.INDEX).equals( params.get(Attrs.WHILE_INDEX))) { i++; } } } //節點處理之後做一些啥事 node.after(currParams, globalParams, ele1, sb); // 回收當前作用域參數 params.clear(); params = null; } } /** * 處理文本替換掉#{item}這種參數 * @param str * @param params * @return * @throws Exception */ private String handText(String str, Map<String, Object> params,Map<String, Object> globalParams) throws Exception { //獲取foreach這種標簽中用於記錄迴圈的變數 String indexStr = MapUtils.getString(params, Attrs.WHILE_INDEX); Integer index = null; if(StringUtils.isNotEmpty(indexStr)) { index = MapUtils.getInteger(params, indexStr); } //匹配#{a}這種參數 String reg1 = "(#\\{)(\\w+)(\\})"; //匹配${a}這種參數 String reg2 = "(\\$\\{)(\\w+)(\\})"; Pattern p1 = Pattern.compile(reg1); Matcher m1 = p1.matcher(str); Pattern p2 = Pattern.compile(reg2); Matcher m2 = p2.matcher(str); String whileList = MapUtils.getString(params, Attrs.WHILE_LIST); Map<String,Object> allParams = getAllParams(params, globalParams); while(m1.find()) { String tmpKey = m1.group(2); String key = whileList == null?tmpKey:(whileList+"_"+tmpKey); key = index == null?key:(key+index); String reKey = "#{"+key+"}"; //如果在foreach類似的迴圈里,可能需要將參數#{xx}替換成#{xx_0},#{xx_1} str = str.replace(m1.group(0), reKey); currParams.put(key, allParams.get(tmpKey)); } while(m2.find()) { String tmpKey = m2.group(2); Object value = allParams.get(tmpKey); if(value != null) { str = str.replace(m2.group(0), getValue(value)); } } return str; } private String getValue(Object value) { String result = ""; if(value instanceof Date) { SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); result = sdf.format((Date)value); } else { result = String.valueOf(value); } return result; } private Map<String, Object> getAllParams(Map<String, Object> currParams, Map<String, Object> globalParams) { Map<String,Object> allParams = new HashMap<String,Object>(); allParams.putAll(globalParams); allParams.putAll(currParams); return allParams; } // 解析一個xml元素 private TempVal parserOneElement(Map<String, Object> currParams, Map<String, Object> globalParams, Element ele, StringBuilder sb) throws Exception { //獲取xml標簽名 String eleName = ele.getName(); //解析一個節點後是否繼續,如遇到if這種節點,就需要判斷test里是否為空 boolean isContinue = false; //聲明一個抽象節點 BaseNode node = null; if (StringUtils.isNotEmpty(eleName)) { //使用節點工廠根據節點名得到一個節點對象比如是if節點還是foreach節點 node = NodeFactory.create(eleName); //解析一下這個節點,返回是否還需要解析節點里的內容 isContinue = node.parse(currParams, globalParams, ele, sb); } return new TempVal(isContinue, ele, node); } public Map<String, Object> getParams() { return currParams; } /** * 封裝一個xml元素被解析後的結果 * @author rongdi */ final static class TempVal { private boolean isContinue; private Element ele; private BaseNode node; public TempVal(boolean isContinue, Element ele, BaseNode node) { this.isContinue = isContinue; this.ele = ele; this.node = node; } public boolean isContinue() { return isContinue; } public void setContinue(boolean isContinue) { this.isContinue = isContinue; } public Element getEle() { return ele; } public void setEle(Element ele) { this.ele = ele; } public BaseNode getNode() { return node; } public void setNode(BaseNode node) { this.node = node; } } }
import org.dom4j.Element; import java.util.HashMap; import java.util.Map; /** * 抽象節點 * @author rongdi */ public abstract class BaseNode { public abstract boolean parse(Map<String, Object> currParams, Map<String, Object> globalParams, Element ele,StringBuilder sb) throws Exception; public void pre(Map<String, Object> currParams,Map<String, Object> globalParams,Element ele,StringBuilder sb) throws Exception { } public void after(Map<String, Object> currParams,Map<String, Object> globalParams,Element ele,StringBuilder sb) throws Exception { } protected Map<String, Object> getAllParams(Map<String, Object> currParams, Map<String, Object> globalParams) { Map<String,Object> allParams = new HashMap<String,Object>(); allParams.putAll(globalParams); allParams.putAll(currParams); return allParams; } }
import java.util.Map; import ognl.Ognl; import org.apache.commons.lang.StringUtils; import org.dom4j.Element; /** * if節點 * @author rongdi */ public class IfNode extends BaseNode{ @Override public boolean parse(Map<String, Object> currParams, Map<String, Object> globalParams, Element ele,StringBuilder sb) throws Exception { //得到if節點的test屬性 String testStr = ele.attributeValue("test"); boolean test = false; try { if(StringUtils.isNotEmpty(testStr)) { //合併全局變數和局部變數 Map<String, Object> allParams = getAllParams(currParams,globalParams); //使用ognl判斷true或者false test = (Boolean) Ognl.getValue(testStr,allParams); } } catch (Exception e) { e.printStackTrace(); throw new Exception("判斷操作參數"+testStr+"不合法"); } if(ele.content() != null && ele.content().size()==0) { test = true; } return test; } }
import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.Set; import ognl.Ognl; import org.apache.commons.collections.MapUtils; import org.apache.commons.lang.StringUtils; import org.dom4j.Element; /** foreach節點屬性如下 collection 需要遍歷的集合 item 遍歷集合後每個元素存放的變數 index 遍歷集合的索引數如0,1,2... separator 遍歷後以指定分隔符拼接 open 遍歷後拼接開始的符號如 ( close 遍歷後拼接結束的符號如 ) */ public class ForeachNode extends BaseNode { @Override public boolean parse(Map<String, Object> currParams, Map<String, Object> globalParams, Element ele, StringBuilder sb) throws Exception { String conditionStr = null; String collectionStr = ele.attributeValue("collection"); String itemStr = ele.attributeValue("item"); String index = ele.attributeValue("index"); String separatorStr = ele.attributeValue("separator"); String openStr = ele.attributeValue("open"); String closeStr = ele.attributeValue("close"); if(StringUtils.isEmpty(index)) { index = "index"; } if(StringUtils.isEmpty(separatorStr)) { separatorStr = ","; } if(StringUtils.isNotEmpty(openStr)) { currParams.put(Attrs.WHILE_OPEN,openStr); } if(StringUtils.isNotEmpty(closeStr)) { currParams.put(Attrs.WHILE_CLOSE,closeStr); } if(StringUtils.isNotEmpty(collectionStr)) { currParams.put(Attrs.WHILE_LIST,collectionStr); } currParams.put(Attrs.WHILE_SEPARATOR,separatorStr); if(index != null) { /** * 如果局部變數中存在當前迴圈變數的值,就表示已經不是第一次進入迴圈標簽了,移除掉開始標記 * 並將局部變數值加1 */ if(currParams.get(index) != null) { currParams.remove(Attrs.WHILE_START); currParams.put(index+"_", (Integer)currParams.get(index+"_") + 1); } else { //第一次進入迴圈標簽內 currParams.put(Attrs.WHILE_START,true); currParams.put(index+"_", 0); } currParams.put(index, (Integer)currParams.get(index+"_")); } boolean condition = true; Map<String, Object> allParams = getAllParams(currParams,globalParams); Object collection = null; if(StringUtils.isNotEmpty(collectionStr)) { //得到待迴圈的集合 collection = Ognl.getValue(collectionStr,allParams); //如果集合屬性不為空,但是條件為null則預設加上一個邊界條件 if(StringUtils.isEmpty(conditionStr)) { //這裡只是用集合演示一下,也可以再加上數組,只不過改成.length而已 if(collection instanceof List) { conditionStr = index+"_<"+collectionStr+".size()"; } else if(collection instanceof Map){ Map map = (Map)collection; Set set = map.entrySet(); List list = new ArrayList(set); allParams.put("_list_", list); conditionStr = index+"_<_list_"+".size()"; } } } currParams.remove(Attrs.WHILE_END); if(StringUtils.isNotEmpty(conditionStr)) { //計算條件的值 condition = (Boolean)Ognl.getValue(conditionStr,allParams); Map<String,Object> tempMap = new HashMap<>(); tempMap.putAll(allParams); tempMap.put(index+"_",(Integer)currParams.get(index+"_") + 1); currParams.put(Attrs.WHILE_END,!(Boolean)Ognl.getValue(conditionStr,tempMap)); } boolean flag = true; currParams.put(Attrs.WHILE_INDEX, index); currParams.put(Attrs.WHILE_FLAG, true); if(condition) { try { if(StringUtils.isNotEmpty(itemStr) && StringUtils.isNotEmpty(collectionStr)) { Object value = null; int idx = Integer.parseInt(currParams.get(index+"_").toString()); if(collection instanceof List) { value = ((List)collection).get(idx); currParams.put(itemStr, value); } else if(collection instanceof Map){ Map map = (Map)collection; Set<Map.Entry<String,Object>> set = map.entrySet(); List<Map.Entry<String,Object>> list = new ArrayList(set); currParams.put(itemStr, list.get(idx).getValue()); currParams.put(index, list.get(idx).getKey()); } } } catch (Exception e) { throw new Exception("從集合或者映射取值"+currParams.get(index)+"錯誤"+e.getMessage()); } } else { flag = false; destroyVars(currParams, index, itemStr); } return flag; } /** * 如果是第一次進入迴圈標簽,則拼上open的內容 */ @Override public void pre(Map<String, Object> currParams, Map<String, Object> globalParams, Element ele, StringBuilder sb) throws Exception { super.pre(currParams, globalParams, ele, sb); boolean start = MapUtils.getBoolean(currParams,Attrs.WHILE_START,false); if(start) { String open = MapUtils.getString(currParams,Attrs.WHILE_OPEN); sb.append(open); } } /** * 如果是最後進入迴圈標簽,則最後拼上close的內容 */ @Override public void after(Map<String, Object> currParams, Map<String, Object> globalParams, Element ele, StringBuilder sb) throws Exception { super.after(currParams, globalParams, ele, sb); boolean end = MapUtils.getBoolean(currParams,Attrs.WHILE_END,false); String separator = MapUtils.getString(currParams,Attrs.WHILE_SEPARATOR); if(!end && StringUtils.isNotEmpty(separator)) { sb.append(separator); } if(end) { String close = MapUtils.getString(currParams,Attrs.WHILE_CLOSE); if(sb.toString().endsWith(separator)) { sb.deleteCharAt(sb.length() - 1); } sb.append(close); } } //釋放臨時變數 private void destroyVars(Map<String, Object> currParams, String index,String varStr) { currParams.remove(Attrs.WHILE_INDEX); currParams.remove(Attrs.WHILE_FLAG); currParams.remove(Attrs.WHILE_SEPARATOR); currParams.remove(Attrs.WHILE_START); currParams.remove(Attrs.WHILE_END); currParams.remove(Attrs.WHILE_LIST);
}
}
import org.dom4j.Element; import java.util.Map; public class SqlNode extends BaseNode{ @Override public boolean parse(Map<String, Object> currParams, Map<String, Object> globalParams, Element ele,StringBuilder sb) throws Exception { return true; } }
import java.util.Arrays; import java.util.List; import java.util.Map; import java.util.concurrent.ConcurrentHashMap; /** * 節點工廠 */ public class NodeFactory { private static Map<String,BaseNode> nodeMap = new ConcurrentHashMap<String,BaseNode>(); private final static List<String> whileList = Arrays.asList("foreach"); static { nodeMap.put("if", new IfNode()); nodeMap.put("sql", new SqlNode()); nodeMap.put("foreach", new ForeachNode()); } public static boolean isWhile(String elementName) { return whileList.contains(elementName); } public static void addNode(String nodeName,BaseNode node) { nodeMap.put(nodeName, node); } public static BaseNode create(String nodeName) { return nodeMap.get(nodeName); } }
/** * 各種標記 * @author rongdi */ public class Attrs { public final static String TRANSACTIONAL = "transactional"; public final static String WHILE_START = "while-start"; public final static String WHILE_END = "while-end"; public final static String WHILE_OPEN = "while-open"; public final static String WHILE_CLOSE = "while-close"; public final static String WHILE_SEPARATOR = "while-separator"; public final static String WHILE_INDEX = "while-index"; public final static String WHILE_FLAG = "while-flag"; public final static String WHILE_LIST = "while-list"; public final static String WHEN_FLAG = "when-flag"; public static final String PROCESS_VAR = "process-var"; public final static String RESULT_FLAG = "result-flag"; public final static String RETURN_FLAG = "return-flag"; public final static String CONSOLE_VAR= "console-var"; public final static String DO = "do"; public final static String INDEX = "index"; public final static String CONDITION = "condition"; public final static String NAME= "name"; public final static String VALUE= "value"; public static final String TYPE = "type"; public static final String FORMAT = "format"; public static final String IF = "if"; public static final String ELSE = "else"; public final static String FILE= "file"; public static final String DATE = "date"; public static final String NOW = "now"; public static final String DECIMAL = "decimal"; public static final String ID = "id"; public static final String PARAMS = "params"; public static final String TARGET = "target"; public static final String SINGLE = "single"; public static final String PAGING = "paging"; public static final String DESC = "desc"; public static final String BREAK = "break"; public static final String CONTINUE = "continue"; public static final String COLLECTION = "collection"; public static final String VAR = "var"; public static final String EXECUTOR = "executor-1"; public static