發表文章

目前顯示的是 2019的文章

ubutun 16.04 安裝 spark

1.安裝 java jdk    安裝 openjdk8 , 因為 Oracle Java 有收費的疑慮 sudo apt-get install default-jdk 2.安裝 Scala    因為 Spark is pre-built with Scala 2.11 except version 2.4.2, which is pre-built with Scala 2.12.    所以安裝 ubutun 預設 Scala  2.11.6   sudo apt-get install scala 3.安裝 SBT   echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823 sudo apt-get update sudo apt-get install sbt   4.安裝SPARK     https://spark.apache.org/downloads.html wget http://apache.stu.edu.tw/spark/spark-2.4.3/spark-2.4.3-bin-hadoop2.7.tgz mkdir spark tar zxvf spark-2.4.3-bin-hadoop2.7.tgz -C spark #mv spark-2.4.3-bin-hadoop2.7/ spark sudo mv spark/ /usr/lib/ cd /usr/lib/spark/spark-2.4.3-bin-hadoop2.7/conf/ cp spark-env.sh.template spark-env.sh 5. 修改環境變數 vi ~/.bashrc e...

nodebb 社群分享外掛( FB, Line, Twitter, Linkedin, Weibo, Weichat. )

圖片
Social Icons Plugin for NodeBB nodebb 社群分享外掛(  FB, Line, Twitter, Linkedin, Weibo, Weichat.  ) Displays social sharing icons beneath every post in a topic for FB, Line, Twitter, Linkedin, Weibo, Weichat. Thanks https://github.com/psychobunny/nodebb-plugin-share-post-icons https://github.com/0312birdzhang/nodebb-plugin-share-post-icons-cn 截图 修改後原始碼 https://github.com/dev-ra/nodebb-plugin-share-post-icons-dropdown

WooCommerce + woocommerce-checkout-fields-for-TW.php + 綠界物流,超商取貨結帳資訊清空問題

修改綠界物流外掛 1.js/ECPay-shipping-checkout.js //取代每次選帳單地址就更新,改在點選超商地圖時才更新 jQuery("#ECPayForm").submit(function() {     jQuery('#billing_postcode').trigger('update');     return true; });      var ecpay_checkout_form = {         .....................         // 取得結帳資訊         get_input_value: function() {             var billing_first_name  = ecpay_checkout_form.ecpay_get_element_value('billing_first_name');             var billing_last_name   = ecpay_checkout_form.ecpay_get_element_value('billing_last_name');             var billing_company     = ecpay_checkout_form.ecpay_get_element_value('billing_company');             var billing_company_tax_id     = ecpay_checkout_form.ecpay_get_element_value('billing_company_tax_i...

spark 讀取 json 檔案,並轉換型態為 ml vector

Spark 讀取 json 檔案,並轉換型態為 ml vector 讀取檔案帶入schema 初始型態為 mllib vector 再利用 mllib vector asML 轉換為 ml vector import org.apache.spark.mllib.linalg.VectorUDT import org.apache.spark.sql.types.{LongType , StructField , StructType} val schema = StructType ( Seq ( StructField ( "post_id" , LongType , true ) , StructField ( "text_w2v" , new VectorUDT , true ))) val w2vdf = spark.read.schema(schema).json(path) .map{ case Row (post_id: Long , text_w2v: org.apache.spark.mllib.linalg.Vector) => (post_id , text_w2v.asML) }.toDF( "post_id" , "text_w2v" ) 參考文件: 1. writing Spark Dataframe to JSON loses format for MLLIB Sparse Vecto r 2. 如何转换org.apache.spark.mllib.linalg.SparseVector至org.apache.spark.ml.linalg.SparseVector?