Java curator操作zookeeper獲取kafka

来源:http://www.cnblogs.com/xinxiucan/archive/2017/08/02/7272357.html
-Advertisement-
Play Games

原文地址：http://blogxinxiucan.sh1.newtouch.com/2017/08/01/Java-curator%E6%93%8D%E4%BD%9Czookeeper%E8%8E%B7%E5%8F%96kafka/ Curator是Netflix公司開源的一個Zookeeper... ...


Java curator操作zookeeper獲取kafka

Curator是Netflix公司開源的一個Zookeeper客戶端，與Zookeeper提供的原生客戶端相比，Curator的抽象層次更高，簡化了Zookeeper客戶端的開發量。

原文地址：http://blogxinxiucan.sh1.newtouch.com/2017/08/01/Java-curator操作zookeeper獲取kafka/

Curator的Maven依賴

    <dependency>
        <groupId>org.apache.curator</groupId>
        <artifactId>curator-recipes</artifactId>
        <version>2.3.0</version>
    </dependency>

本地啟動kafka

啟動zookeeper

./bin/zookeeper-server-start.sh config/zookeeper.properties

啟動kafka

./bin/kafka-server-start.sh config/server.properties

啟動kafkaManager

sudo ./bin/kafka-manager

代碼

項目結構

先看測試

package com.xinxiucan.test;

import java.util.List;

import com.xinxiucan.api.BrokerApi;
import com.xinxiucan.api.TopicApi;
import com.xinxiucan.zookeeper.ZKBean;

public class AppTest {

    public static void main(String[] args) {
        ZKBean zk = new ZKBean();
        zk.setZkURL("127.0.0.1:2181");
        List<String> ids = BrokerApi.findBrokerIds(zk);
        for (String id : ids) {
            String idInfo = BrokerApi.findBrokerById(zk, id);
            System.out.println(idInfo);
        }

        List<String> topicNames = TopicApi.findAllTopicName(zk);
        for (String topicName : topicNames) {
            System.out.println("topicName:" + topicName);
        }
        
        List<String> topics = TopicApi.findAllTopics(zk);
        for (String topic : topics) {
            System.out.println("topic:" + topic);
        }

    }
}

運行結果

{"jmx_port":-1,"timestamp":"1501636761404","endpoints":["PLAINTEXT://can:9092"],"host":"can","version":3,"port":9092}
topicName:test_xin_1
topicName:xin
topic:{"version":1,"partitions":{"0":[0]}}
topic:{"version":1,"partitions":{"0":[0]}}

剩餘代碼

ZKBean

package com.xinxiucan.zookeeper;

import java.io.Serializable;

public class ZKBean implements Serializable {

    private static final long serialVersionUID = -6057956208558425192L;

    private int id = -1;

    private String name;

    private String zkURL;

    private String version = "0.8.2.2";

    private boolean jmxEnable;

    private String jmxAuthUsername;

    private String jmxAuthPassword;

    private boolean jmxWithSsl;

    private int zkConnectionTimeout = 30;

    private int maxRetryCount = 3;

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getZkURL() {
        return zkURL;
    }

    public void setZkURL(String zkURL) {
        this.zkURL = zkURL;
    }

    public boolean isJmxEnable() {
        return jmxEnable;
    }

    public void setJmxEnable(boolean jmxEnable) {
        this.jmxEnable = jmxEnable;
    }

    public String getJmxAuthUsername() {
        return jmxAuthUsername;
    }

    public void setJmxAuthUsername(String jmxAuthUsername) {
        this.jmxAuthUsername = jmxAuthUsername;
    }

    public String getJmxAuthPassword() {
        return jmxAuthPassword;
    }

    public void setJmxAuthPassword(String jmxAuthPassword) {
        this.jmxAuthPassword = jmxAuthPassword;
    }

    public boolean isJmxWithSsl() {
        return jmxWithSsl;
    }

    public void setJmxWithSsl(boolean jmxWithSsl) {
        this.jmxWithSsl = jmxWithSsl;
    }

    public int getZkConnectionTimeout() {
        return zkConnectionTimeout;
    }

    public void setZkConnectionTimeout(int zkConnectionTimeout) {
        this.zkConnectionTimeout = zkConnectionTimeout;
    }

    public int getMaxRetryCount() {
        return maxRetryCount;
    }

    public void setMaxRetryCount(int maxRetryCount) {
        this.maxRetryCount = maxRetryCount;
    }

    public String getVersion() {
        return version;
    }

    public void setVersion(String version) {
        this.version = version;
    }

    @Override
    public int hashCode() {
        return this.id;
    }

    @Override
    public boolean equals(Object obj) {
        if (obj == null) {
            return false;
        }
        if (this == obj) {
            return true;
        }
        if (obj instanceof ZKBean) {
            ZKBean anotherZKCluster = (ZKBean) obj;
            return this.id == anotherZKCluster.getId();
        } else {
            return false;
        }
    }

}

ZKConnectionFactory

package com.xinxiucan.zookeeper;

import org.apache.curator.RetryPolicy;
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.retry.ExponentialBackoffRetry;

public class ZKConnectionFactory {

    public static CuratorFramework buildCuratorFramework(ZKBean zk) {
        RetryPolicy retryPolicy = new ExponentialBackoffRetry(zk.getZkConnectionTimeout() * 1000, zk.getMaxRetryCount());
        CuratorFramework client = CuratorFrameworkFactory.newClient(zk.getZkURL(), retryPolicy);
        client.start();
        return client;
    }

}

BrokerApi

package com.xinxiucan.api;

import java.util.HashMap;
import java.util.List;
import java.util.Map;

import org.apache.curator.framework.CuratorFramework;

import com.xinxiucan.KafkaZKPath;
import com.xinxiucan.zookeeper.ZKBean;
import com.xinxiucan.zookeeper.ZKConnectionFactory;

/**
 * 獲取broker信息
 * @author xinxiucan
 */
public class BrokerApi {

    /**
     * 根據zk獲取Broker ID列表
     * @param zkBean
     * @return
     */
    public static List<String> findBrokerIds(ZKBean zkBean) {
        CuratorFramework client = null;
        try {
            client = ZKConnectionFactory.buildCuratorFramework(zkBean);
            List<String> brokerIdStrs = client.getChildren().forPath(KafkaZKPath.BROKER_IDS_PATH);
            return brokerIdStrs;
        } catch (Exception e) {
            throw new RuntimeException(e);
        } finally {
            if (client != null) {
                client.close();
            }
        }
    }

    /**
     * 根據zk&brokerId獲取Broker信息
     * @param zkBean
     * @return
     */
    public static String findBrokerById(ZKBean zkBean, String id) {
        CuratorFramework client = null;
        try {
            client = ZKConnectionFactory.buildCuratorFramework(zkBean);
            String brokerIdPath = KafkaZKPath.getBrokerSummeryPath(id);
            String brokerInfo = new String(client.getData().forPath(brokerIdPath), "UTF-8");
            return brokerInfo;
        } catch (Exception e) {
            throw new RuntimeException(e);
        } finally {
            if (client != null) {
                client.close();
            }
        }
    }

    public static Map<String, String> findAllBrokerSummarys(ZKBean zkBean) {
        CuratorFramework client = null;
        try {
            client = ZKConnectionFactory.buildCuratorFramework(zkBean);
            List<String> brokerIds = client.getChildren().forPath(KafkaZKPath.BROKER_IDS_PATH);
            if (brokerIds == null || brokerIds.size() == 0) {
                return null;
            }
            Map<String, String> brokerMap = new HashMap<String, String>();
            for (String brokerId : brokerIds) {
                String brokerIdPath = KafkaZKPath.getBrokerSummeryPath(brokerId);
                String brokerInfo = new String(client.getData().forPath(brokerIdPath), "UTF-8");
                brokerMap.put(brokerId, brokerInfo);
            }
            return brokerMap;
        } catch (Exception e) {
            throw new RuntimeException(e);
        } finally {
            if (client != null) {
                client.close();
            }
        }
    }

}

TopicApi

package com.xinxiucan.api;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import org.apache.curator.framework.CuratorFramework;
import org.apache.zookeeper.data.Stat;

import com.xinxiucan.KafkaZKPath;
import com.xinxiucan.ReplicasAssignmentBuilder;
import com.xinxiucan.util.Object2Array;
import com.xinxiucan.zookeeper.ZKBean;
import com.xinxiucan.zookeeper.ZKConnectionFactory;


/**
 * 獲取Topic信息
 * @author can
 */
public class TopicApi {

    /**
     * 創建Topic
     * @param zkBean
     * @param topic
     * @param partitionsCount
     * @param replicationFactorCount
     * @param topicConfig
     */
    public static void createTopic(ZKBean zkBean, String topic, int partitionsCount, int replicationFactorCount,
            Map<String, String> topicConfig) {
        CuratorFramework client = null;
        try {
            client = ZKConnectionFactory.buildCuratorFramework(zkBean);
            List<String> brokerIds = client.getChildren().forPath(KafkaZKPath.BROKER_IDS_PATH);
            List<Integer> ids = new ArrayList<>();
            for (String str : brokerIds) {
                int i = Integer.parseInt(str);
                ids.add(i);
            }
            Map<String, List<Integer>> replicasAssignment = ReplicasAssignmentBuilder.assignReplicasToBrokers(ids, partitionsCount,
                    replicationFactorCount);
            Map<String, Object> topicSummaryMap = new HashMap<String, Object>();
            topicSummaryMap.put("version", 1);
            topicSummaryMap.put("partitions", replicasAssignment);
            byte[] topicSummaryData = Object2Array.objectToByteArray(topicSummaryMap);
            String topicSummaryPath = KafkaZKPath.getTopicSummaryPath(topic);
            Stat stat = client.checkExists().forPath(topicSummaryPath);
            if (stat == null) {
                // create
                client.create().creatingParentsIfNeeded().forPath(topicSummaryPath, topicSummaryData);
            } else {
                // update
                client.setData().forPath(topicSummaryPath, topicSummaryData);
            }

            Map<String, Object> topicConfigMap = new HashMap<String, Object>();
            topicConfigMap.put("version", 1);
            topicConfigMap.put("config", topicConfig);
            byte[] topicConfigData = Object2Array.objectToByteArray(topicConfigMap);

            String topicConfigPath = KafkaZKPath.getTopicConfigPath(topic);
            Stat stat2 = client.checkExists().forPath(topicConfigPath);
            if (stat2 == null) {
                // create
                client.create().creatingParentsIfNeeded().forPath(topicConfigPath, topicConfigData);
            } else {
                // update
                client.setData().forPath(topicConfigPath, topicConfigData);
            }
        } catch (Exception e) {
            throw new RuntimeException(e);
        } finally {
            if (client != null) {
                client.close();
            }
        }
    }

    /**
     * 刪除Topic
     * @param zkBean
     * @param topicName
     */
    public static void deleteTopic(ZKBean zkBean, String topicName) {
        CuratorFramework client = null;
        try {
            client = ZKConnectionFactory.buildCuratorFramework(zkBean);
            String adminDeleteTopicPath = KafkaZKPath.getDeleteTopicPath(topicName);
            client.create().creatingParentsIfNeeded().forPath(adminDeleteTopicPath, null);
        } catch (Exception e) {
            throw new RuntimeException(e);
        } finally {
            if (client != null) {
                client.close();
            }
        }
    }

    /**
     * 查詢topic
     * @param zkBean
     * @return
     */
    public static List<String> findAllTopics(ZKBean zkBean) {
        CuratorFramework client = null;
        try {
            client = ZKConnectionFactory.buildCuratorFramework(zkBean);
            List<String> topicNames = client.getChildren().forPath(KafkaZKPath.BROKER_TOPICS_PATH);
            List<String> topicSummarys = new ArrayList<String>();
            for (String topicName : topicNames) {
                byte[] topicData = client.getData().forPath(KafkaZKPath.getTopicSummaryPath(topicName));
                topicSummarys.add(new String(topicData, "UTF-8"));
            }
            // add delete flag
            List<String> deleteTopics = client.getChildren().forPath(KafkaZKPath.ADMIN_DELETE_TOPICS_PATH);
            return topicSummarys;
        } catch (Exception e) {
            throw new RuntimeException(e);
        } finally {
            if (client != null) {
                client.close();
            }
        }
    }

    public static List<String> findAllTopicName(ZKBean zkBean) {
        CuratorFramework client = null;
        try {
            client = ZKConnectionFactory.buildCuratorFramework(zkBean);
            return client.getChildren().forPath(KafkaZKPath.BROKER_TOPICS_PATH);
        } catch (Exception e) {
            throw new RuntimeException(e);
        } finally {
            if (client != null) {
                client.close();
            }
        }
    }

    
}

KafkaZKPath

package com.xinxiucan;

public class KafkaZKPath {

    public static final String COMSUMERS_PATH = "/consumers";
    public static final String BROKER_IDS_PATH = "/brokers/ids";
    public static final String BROKER_TOPICS_PATH = "/brokers/topics";
    public static final String CONFIG_TOPICS_PATH = "/config/topics";
    public static final String CONFIG_CHANGES_PATH = "/config/changes";
    public static final String CONTROLLER_PATH = "/controller";
    public static final String CONTROLLER_EPOCH_PATH = "/controller_epoch";
    public static final String ADMIN_PATH = "/admin";
    public static final String ADMIN_REASSGIN_PARTITIONS_PATH = "/admin/reassign_partitions";
    public static final String ADMIN_DELETE_TOPICS_PATH = "/admin/delete_topics";
    public static final String ADMIN_PREFERED_REPLICA_ELECTION_PATH = "/admin/preferred_replica_election";

    public static String getTopicSummaryPath(String topic) {
        return BROKER_TOPICS_PATH + "/" + topic;
    }

    public static String getTopicPartitionsStatePath(String topic,String partitionId) {
        return getTopicSummaryPath(topic) + "/partitions/" + partitionId+ "/state";
    }

    public static String getTopicConfigPath(String topic) {
        return CONFIG_TOPICS_PATH + "/" + topic;
    }

    public static String getDeleteTopicPath(String topic) {
        return ADMIN_DELETE_TOPICS_PATH + "/" + topic;
    }
    
    public static String getBrokerSummeryPath(String id) {
        return BROKER_IDS_PATH + "/" + id;
    }
    
    public static String getConsumerOffsetPath(String groupId) {
        return  COMSUMERS_PATH + "/" + groupId +"/offsets";
    }
    
    public static String getTopicConsumerOffsetPath(String groupId,String topic){
        return  COMSUMERS_PATH + "/" + groupId +"/offsets/"+topic;
    }
    
    public static String getTopicPartitionConsumerOffsetPath(String groupId, String topic,String partitionId){
        return  COMSUMERS_PATH + "/" + groupId +"/offsets/"+topic + "/" + partitionId;
    }
    
    public static String getTopicPartitionConsumerOwnerPath(String groupId, String topic, String partitionId) {
        return  COMSUMERS_PATH + "/" + groupId +"/owners/"+topic + "/" + partitionId;
    }
    
    
}

ReplicasAssignmentBuilder

package com.xinxiucan;

import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Random;

public class ReplicasAssignmentBuilder {

    public static Map<String, List<Integer>> assignReplicasToBrokers(List<Integer> brokerListSet, int nPartitions, int nReplicationFactor) {
        return assignReplicasToBrokers(brokerListSet, nPartitions, nReplicationFactor, -1, -1);
    }

    /**
     * There are 2 goals of replica assignment:
     * 1. Spread the replicas evenly among brokers.
     * 2. For partitions assigned to a particular broker, their other replicas are spread over the other brokers.
     *
     * To achieve this goal, we:
     * 1. Assign the first replica of each partition by round-robin, starting from a random position in the broker list.
     * 2. Assign the remaining replicas of each partition with an increasing shift.
     *
     * Here is an example of assigning
     * broker-0  broker-1  broker-2  broker-3  broker-4
     * p0        p1        p2        p3        p4       (1st replica)
     * p5        p6        p7        p8        p9       (1st replica)
     * p4        p0        p1        p2        p3       (2nd replica)
     * p8        p9        p5        p6        p7       (2nd replica)
     * p3        p4        p0        p1        p2       (3nd replica)
     * p7        p8        p9        p5        p6       (3nd replica)
     */
    public static Map<String, List<Integer>> assignReplicasToBrokers(List<Integer> brokerListSet, int nPartitions, int nReplicationFactor,
            int fixedStartIndex, int startPartitionId) {
        // sort
        brokerListSet.sort(new Comparator<Integer>() {

            @Override
            public int compare(Integer o1, Integer o2) {
                return o1 - o2;
            }

        });
        if (nPartitions <= 0) {
            throw new RuntimeException("num of partition cannot less than 1");
        }
        if (nReplicationFactor <= 0) {
            throw new RuntimeException("num of replication factor cannot less than 1");
        }
        if (nReplicationFactor > brokerListSet.size()) {
            throw new RuntimeException("broker num is less than replication factor num");
        }

        Random random = new Random();

        Map<String, List<Integer>> result = new HashMap<String, List<Integer>>();
        int startIndex = fixedStartIndex >= 0 ? fixedStartIndex : random.nextInt(brokerListSet.size());
        int currentPartitionId = startPartitionId >= 0 ? startPartitionId : 0;
        int nextReplicaShift = fixedStartIndex >= 0 ? fixedStartIndex : random.nextInt(brokerListSet.size());

        for (int i = 0; i < nPartitions; i++) {
            if (currentPartitionId > 0 && ((currentPartitionId % brokerListSet.size()) == 0)) {
                nextReplicaShift++;
            }
            int firstReplicaIndex = (currentPartitionId + startIndex) % brokerListSet.size();
            List<Integer> replicaList = new ArrayList<Integer>();
            replicaList.add(brokerListSet.get(firstReplicaIndex));
            for (int j = 0; j < (nReplicationFactor - 1); j++) {
                replicaList.add(brokerListSet.get(getReplicaIndex(firstReplicaIndex, nextReplicaShift, j, brokerListSet.size())));
            }
            result.put("" + currentPartitionId, getReverseList(replicaList));
            currentPartitionId++;
        }

        return result;
    }

    private static List<Integer> getReverseList(List<Integer> list) {
        List<Integer> result = new ArrayList<Integer>();
        for (int i = list.size() - 1; i >= 0; i--) {
            result.add(list.get(i));
        }
        return result;
    }

    private static int getReplicaIndex(int firstReplicaIndex, int nextReplicaShift, int replicaIndex, int nBrokers) {

        int shift = 1 + (nextReplicaShift + replicaIndex) % (nBrokers - 1);
        int result = (firstReplicaIndex + shift) % nBrokers;
        // System.out.println(firstReplicaIndex+"|"+nextReplicaShift+"|"+replicaIndex+"="+result);
        return result;
    }

    private static void printAssignReplicas(Map<String, List<Integer>> map, List<Integer> brokerListSet) {
        Map<Integer, List<String>> brokerMap = new HashMap<Integer, List<String>>();
        for (int i = 0; i < brokerListSet.size(); i++) {
            List<String> partitions = new ArrayList<String>();
            for (int j = 0; j < map.size(); j++) {
                List<Integer> brokerIds = map.get(j + "");
                if (brokerIds.contains(i)) {
                    partitions.add("" + j);
                }
            }
            brokerMap.put(i, partitions);
        }
    }

}

您的分享是我們最大的動力!

-Advertisement-
Play Games
更多相關文章
  • ionic 項目簽名
    一.ionic 自動簽名的好處與壞處(ionic build android/ios) 好處在於:可以直接安裝手機上進行安裝測試,也可以上傳Android或者iOS平臺 不好的地方在於:你的電腦環境變了換電腦了,再打包出來的簽名就會不一樣了,再打包出來的app就無法覆蓋之前的版本,會報簽名不一樣的問 ...
  • activity啟動模式之standard
    activity啟動模式之standard 一、簡介 這種模式是預設的，不用我們自己設定 就像一隻疊加在棧中 如果退出，就一個個退出，其實就是我們自己用手機的那種感受 二、代碼實例 activityLaunchStandard.MainActivity 每次激活activity都會新建一個activ ...
  • 請求碼和結果碼
    請求碼和結果碼 一、簡介 請求碼： 例如請求頁面有多個button，根據請求碼就知道是哪個button在請求 結果碼： 多個請求可以打開多個頁面，根據結果碼就知道我們打開的是哪個界面 請求碼是用來標識請求源的，結果碼是用來標識結果源的。 二、具體步驟 這裡演示結果碼的 1、界面1裡面的結果碼是100 ...
  • 從Activity中返回數據
    從Activity中返回數據 一、簡介 這裡也就是使用intent方式返回數據。 二、具體步驟 在MainActivity通過一個button訪問Activity01頁面，然後將Activity01頁面的數據返回到MainActivity頁面。 調用Activity頁面：MainActivity 被 ...
  • cocoapods 配置
    二、CocoaPods 安裝 CocoaPods可以方便地通過Mac自帶的RubyGems安裝。 打開Terminal（Mac電腦自帶的終端）： （1）.設置ruby的軟體源 這是因為ruby的軟體源rubygems.org因為使用亞馬遜的雲服務，被我天朝屏蔽了，需要更新一下ruby的源，過程如下： ...
  • 【代碼筆記】iOS-NSFileManager
    一，代碼。 二，輸出。 ...
  • [20170728]oracle保留字.txt
    [20170728]oracle保留字.txt--//oracle有許多保留字,我印象最深的就是使用rman備份表空間test,test就是rman裡面的保留字.--//還有rman也是rman裡面的保留字.如果在應用中儘量規避不要使用這些保留字.--//探究一下,oracle內部是否也會不小心這些 ...
  • Oracle01——基本查詢、過濾和排序、單行函數、多行函數和多表查詢
    作者： kent鵬 轉載請註明出處： http://www.cnblogs.com/xieyupeng/p/7272236.html Oracle的集群 Oracle的體繫結構 SQL> --當前用戶 SQL> show user USER 為 "SCOTT" SQL> --當前用戶下的表 SQL> ...
一周排行
    -Advertisement-
    Play Games
  • 移動開發（一）：使用.NET MAUI開發第一個安卓APP
    移動開發（一）：使用.NET MAUI開發第一個安卓APP 對於工作多年的C#程式員來說，近來想嘗試開發一款安卓APP，考慮了很久最終選擇使用.NET MAUI這個微軟官方的框架來嘗試體驗開發安卓APP，畢竟是使用Visual Studio開發工具，使用起來也比較的順手，結合微軟官方的教程進行了安卓 ...
  • wpf ToggleButton選中效果和一個登錄界面
    前言 QuestPDF 是一個開源 .NET 庫，用於生成 PDF 文檔。使用了C# Fluent API方式可簡化開發、減少錯誤並提高工作效率。利用它可以輕鬆生成 PDF 報告、發票、導出文件等。 項目介紹 QuestPDF 是一個革命性的開源 .NET 庫，它徹底改變了我們生成 PDF 文檔的方 ...
  • 彈幕樹洞項目功能新增篇
    項目地址 項目後端地址： https://github.com/ZyPLJ/ZYTteeHole 項目前端頁面地址： ZyPLJ/TreeHoleVue (github.com) https://github.com/ZyPLJ/TreeHoleVue 目前項目測試訪問地址： http://tree ...
  • 第27篇 sqlserver2022詳細安裝步驟
    話不多說，直接開乾 一.下載 1.官方鏈接下載： https://www.microsoft.com/zh-cn/sql-server/sql-server-downloads 2.在下載目錄中找到下麵這個小的安裝包 SQL2022-SSEI-Dev.exe，運行開始下載SQL server； 二. ...
  • .NET 開源高性能 MQTT 類庫
    前言 隨著物聯網（IoT）技術的迅猛發展，MQTT（消息隊列遙測傳輸）協議憑藉其輕量級和高效性，已成為眾多物聯網應用的首選通信標準。 MQTTnet 作為一個高性能的 .NET 開源庫，為 .NET 平臺上的 MQTT 客戶端與伺服器開發提供了強大的支持。 本文將全面介紹 MQTTnet 的核心功能 ...
  • Serilog文檔翻譯系列（六） - 可用的接收器、增強器、格式化輸出
    Serilog支持多種接收器用於日誌存儲，增強器用於添加屬性，LogContext管理動態屬性，支持多種輸出格式包括純文本、JSON及ExpressionTemplate。還提供了自定義格式化選項，適用於不同需求。 ...
  • 警惕 Visual Studio 屬性求值副作用導致邏輯不符合預期
    目錄簡介獲取 HTML 文檔解析 HTML 文檔測試參考文章 簡介 動態內容網站使用 JavaScript 腳本動態檢索和渲染數據，爬取信息時需要模擬瀏覽器行為，否則獲取到的源碼基本是空的。 本文使用的爬取步驟如下： 使用 Selenium 獲取渲染後的 HTML 文檔 使用 HtmlAgility ...
  • [使用目前最新版]HybridCLR6.9.0+YooAsset2.2.4實現純C# Unity熱更新方案 (一)
    1.前言 什麼是熱更新 游戲或者軟體更新時，無需重新下載客戶端進行安裝，而是在應用程式啟動的情況下，在內部進行資源或者代碼更新 Unity目前常用熱更新解決方案 HybridCLR，Xlua，ILRuntime等 Unity目前常用資源管理解決方案 AssetBundles，Addressable， ...
  • 在 ASP.NET Core Web API 中使用操作篩選器統一處理通用操作
    本文章主要是在C# ASP.NET Core Web API框架實現向手機發送驗證碼簡訊功能。這裡我選擇是一個互億無線簡訊驗證碼平臺，其實像阿裡雲，騰訊雲上面也可以。 首先我們先去 互億無線 https://www.ihuyi.com/api/sms.html 去註冊一個賬號 註冊完成賬號後，它會送 ...
  • 第28篇 如何.net中實現高效可靠數據同步api
    通過以下方式可以高效，並保證數據同步的可靠性 1.API設計 使用RESTful設計，確保API端點明確，並使用適當的HTTP方法（如POST用於創建，PUT用於更新）。 設計清晰的請求和響應模型，以確保客戶端能夠理解預期格式。 2.數據驗證 在伺服器端進行嚴格的數據驗證，確保接收到的數據符合預期格 ...
所有分類