Microservice Structure

 如何解决大流水线不好改造的问题

The Abstract & The source

花了两个小时读完microservice,觉得这个设计思路跟罗辑思维罗胖讲的『U盘人』有类似之处,都是自带信息,随时插拔,自由协作,当然还要容错性高。

这个是原文链接,做软件设计的值得多看看,开阔开阔思路:

http://martinfowler.com/articles/microservices.html

The Good:

With a monolith any changes require a full build and deployment of the entire application. With microservices, however, you only need to redeploy the service(s) you modified.

The Bad:

The downside is that you have to worry about changes to one service breaking its consumers. The traditional integration approach is to try to deal with this problem using versioning, but the preference in the microservice world is to only use versioning as a last resort. We can avoid a lot of versioning by designing services to be as tolerant as possible to changes in their suppliers.

The Specification:

Organize around business capabilities

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.

— Melvyn Conway, 1967

对比:

Monolithic的团队注重组织分工,各司其职:

Microservice的团队更加注重business capability:

The Example:

The Microservice is the Evolutionary Design, or not?

The Guardian website is a good example of an application that was designed and built as a monolith, but has been evolving in a microservice direction. The monolith still is the core of the website, but they prefer to add new features by building microservices that use the monolith’s API.

在网页中加上第三方搜索(django框架,bing搜索)

搜索,是一个网站基本的能力。在这个例子中,我们添加了搜索功能,其原理是:

  • 注册Bing Search API
  • 调用Bing Search API,得到结果的JSON
  • 将数据存入一个result列表中
  • 展示result

此教程来自http://www.tangowithdjango.com/book17/chapters/bing_search.html,我将他的意思再总结了一下。下面,我们就来分别看这几步:

1. 注册Bing search API

1.1 注册marketplace

首先需要注册一个Microsoft Azure Marketplace的账号。可以通过Hotmail账号注册 (注意语言需要使用英文。用中文的话,微软是不会提供API的)。

1.2 Sign Up应用

然后是在这个应用商店中找到应用,如链接:https://datamarket.azure.com/dataset/bing/search。微软的搜索引擎按照transaction per month来算的,我们就注册一个5000TPS的即可。

注册完成之后,还需要找到API key,最好是新建一个API key,这个key会在后面调用的时候使用。

2. 调用Bing Search API

后面的这几部都是代码实现的。

2.1 根据API文档配置参数

目前我们先创建一个bing_search.py的文件。然后需要参考微软的API文档,看看这个是怎么调用的。

https://datamarket.azure.com/dataset/bing/search#schema可以找到service_root_url,然后这个作为一个root。

然后查阅《Bing Search API Quick Start and Code Samples》文件,上面有列举the examples of your final URI to the Bing Search API:

https://api.datamarket.azure.com/Bing/Search/Web?$format=json&Query=%27Xbox%27

根据这个,我们定义了几个参数,传入到search_url中:

	search_url = "{0}{1}?$format=json&$top={2}&$skip={3}&Query={4}".format(
        root_url,
        source,
        results_per_page,
        offset,
        query
    )

2.2 通过password_manager来管理密码:

	password_mgr = urllib.request.HTTPPasswordMgrWithDefaultRealm()
    password_mgr.add_password(None, search_url, username, BING_API_KEY)

3. 将数据存入result列表中

3.1 urllib.request.urlopen()的解码处理

这里我们使用的url open得到的response,而url open返回的是一个binary的文件,son又zhi只接受utf-8格式的,所以这里需要将接收到的文件decode为utf-8格式.然后再在json.loads。

	# Convert the bytes to a str. Because "The JSON object must be str, not 'bytes'"
        str_response = response.decode('utf-8')

        # Convert the string response to a Python dictionary object.
        json_response = json.loads(str_response)

3.2 如何将json对应到python列表中

下面第一句的数组是对JSON的标准处理。关于json的知识,可以参考这篇文章link。简单来说,JSON就是一个标准化的数组格式

	for result in json_response['d']['results']:
            results.append({
                'title': result['Title'],
                'link': result['Url'],
                'summary': result['Description']
            })

如果在PHP中,也是通过类似的方法:

foreach($jsonObj->d->results as $value)

3.3 通过命令行的形式验证

在实施网页版本之前,首先通过命令行的形式保证这个API可以Working。

具体请见代码

4. 展示result

需要在Template中添加相应的search.html

需要在views中添加search(request)的view。

需要在urls.py中添加url

最后在base上添加链接。

5. 代码

import json
import urllib.request
import urllib.parse
from rango.keys import BING_API_KEY



def run_query(search_terms):
    # Specify the base
    root_url = 'https://api.datamarket.azure.com/Bing/Search/v1/'
    source = 'Web'

    # Specify how many results we wish to return per page
    results_per_page = 10
    offset = 0

    # Warp quotes around our query terms as required by the Bing API
    query = "'{0}'".format(search_terms)
    query = urllib.request.quote(query)

    # Constructs the latter part of our request's URL
    search_url = "{0}{1}?$format=json&$top={2}&$skip={3}&Query={4}".format(
        root_url,
        source,
        results_per_page,
        offset,
        query
    )

    # Setup authentication with the Bing servers.
    # The username MUST be a blank string, and put in your API key!
    username = ''

    # Create a 'password manager' which handles authentication for us.
    password_mgr = urllib.request.HTTPPasswordMgrWithDefaultRealm()
    password_mgr.add_password(None, search_url, username, BING_API_KEY)

    # Create our results list which we'll populate
    results = []

    try:
        # Prepare for connecting to Bing's servers.
        handler = urllib.request.HTTPBasicAuthHandler(password_mgr)
        opener = urllib.request.build_opener(handler)
        urllib.request.install_opener(opener)

        # Connect to the server and read the response generated.
        response = urllib.request.urlopen(search_url).read()

        # Convert the bytes to a str. Because "The JSON object must be str, not 'bytes'"
        str_response = response.decode('utf-8')

        # Convert the string response to a Python dictionary object.
        json_response = json.loads(str_response)

        # Loop through each page returned, populating out results list.
        for result in json_response['d']['results']:
            results.append({
                'title': result['Title'],
                'link': result['Url'],
                'summary': result['Description']
            })
    except urllib.request.URLError as e:
        # Catch a URLError exception - something went wrong when connecting
        print("Error when querying the Bing API: "+e)

    return results

def main():
    search_terms = input("Input the word you want to search>> ")
    results = run_query(search_terms)
    print("Rank"+' '+'Title'+50*' '+'URL')
    rank = 0
    for result in results:
        rank += 1
        print(str(rank).ljust(5), end=' ')
        print(result['title'].ljust(50), end=' ')
        print(result['link'])

if __name__ == '__main__':
    main()

Python中不能安装python-docx的问题

python-docx是咱们常用的一个python中处理word文档的库,相对于VB来说,更容易学,也比较适合通过python作为胶水,将其他内容放在word中。然而很多时候安装python-docx会报错,如何解决呢?

0. 前提

必须使用的是windows系统,python-docx在mac os上是不能够工作的。

另外python-docx使用了lxml,而lxml的更新又不是那么快,所以导致了问题。

在命令行中,使用pip install python-docx安装

1. 问题提示

一般来说,大家看到的提示有这么几个:

  1. 提示vcvarsall.bat不存在。
  2. 提示 fatal error C1083: Cannot open include file: ‘libxml/xpath.h’: No such file or directory

2. 问题解决方案

对于上面两个问题,由以下方法解决:

  1. vcvarsall.bat不存在,是因为没有安装visual studio。可以到微软官网下载一个visual studio express并且安装。
  2. 再提示的libxml/xpath.h的错误,通常是因为lxml的安装有问题导致的。所以这里建议分两步安装,先安装lxml的非官方的包:lxml补丁包链接,然后再通过pip install python-docx即可。
    • 注意补丁包的选择,需要选择跟你的python版本对应的版本,例如如果你使用的是32位的python35,就需要选择lxml-3.5.0-cp35-none-win32.whl这个版本
    • whl包的安装:用命令行工具cd到whl目录,再pip install即可