1. Goal

针对机械领域的复杂工程问题提出解决思路，设计满足特定需求的机械、电子系统或具体解决方法与流程

基础知识：

• 工程图学：绘制和阅读机械工程草图
• 刚体力学：刚体运动的数学建模表达，平动、转动、定轴转动
• 机械制图CAD：CAD软件的基本操作，三维设计SolidWorks
• 设计、材料和制造：不同材料和制造的基础知识

扩展知识

• 电机与拖动：直流电机、伺服电机的结构与工作原理，矩、力的计算

机械的设计思维，

• 功能，设计
• 功能性需求：模仿人工怎么采摘，两个手指头往下一揪
• 实际问题：相机平面误差，冗余性设计，宽度做宽一点
• 改进：做一个机构，根据功能来，
• 从概念到实用
• 供应链
• 知道已经有了哪些东西
• 电机，问答机制
• 电机

设计时候的漏洞

• general的经验，可以避免
• spec的经验，很难避免

1. Intro

Goal: manage startup between 2-150 people, responsible for other people’s output

Limit: time and experience

Solution:

• find the smallest possible set of techniques to be a manager
• First principle

2. What is Manager’s Job

Goal: increase the output of the team, the single principle

Limit: the activities of a manager is complicated, and not directly map to business value

How to measure the value of a manager? (Rather than an individual contributor)

• increase the output
• maximize the value of employee

1. Intro

Prerequisite: 将两个单目相机单独标定，得到了两个相机的内参和畸变参数

Goal: 就像人眼一样，需要知道两个相机的位置关系才能判断深度。两个相机并不是完全水平地放置，需要确定两者之间的R、T关系，修正视角使得两个图像中的位置在y方向一致

How：

1. Calibration for stereo camera set
2. Rectification, find R, T to make them aligned in the y-axis

2. 原理

2.1. stereoCalibrate

利用之前标定的两个相机的单独的内参，在标定时fix_intrinsic，只估计R，T，E（Essential），F（Fundamental）between two cameras

minimize re-projection error

2.2. StereoRectify

1. Intro

单目相机标定目标：相机的内参（fx,fy,cx,cy）和畸变，建立从世界坐标到像素坐标的映射

How: 张正友标定法

原理：

1. 从世界坐标到像素坐标的映射包含了内参和外参，是一个$3\times 3$矩阵（单应矩阵$H$），有8个自由度（一个特征点的xy包含了两个方程，所以一共需要4个特征点）
2. 求出单应矩阵之后再分别求内参、外参

应用：通过棋盘格，棋盘格设为世界坐标系，并且边长也知道，因此世界坐标系的点坐标知道了，因为特征点多于4个，因此可以做优化

2. 推导

利用单应矩阵的性质

1. Intro

给定目标在某一帧内的状态，估计目标在后续帧中的状态，三个方面

• 找到：在某个位置，颜色过滤
• 感兴趣物体：定义一些特征，表达、提取特征
• 后续帧：找到最像的物体，最重要的子问题

• 运动模型输入的是两帧，输出候选框
• 特征模型输出候选框+特征
• 观测模型：选择最像的

2. 运动模型

如何表达位置？Bounding box

如何生成候选框？sliding window

3. 特征模型

• 像素值
• 统计 hist
• 变换 梯度

1. Intro

Web realtime trans

differ from web sockets

• reduce the latency

2. How to communicate

2.1. Signaling

A,B connect to server and exchange informations

2.2. How?

API

Indentity

1. Choosing Your Opportunity

Opportunity is neither a matter of simple luck nor a certain path. Opportunity is the formulation of a hypothesis by an entrepreneur about how to potentially create and capture value by providing something that is “missing” from the marketplace.

1.1. A Strategic Approach

3 insight about opportunities

• opportunities are everywhere
• while opportunities are everywhere, not every opportunity is the right opportunity for every individual.
• opportunities are uncertain

Opportunity is a hypothesis about the potential for creating and capture value that, by its very nature, is uncertain at the moment it is first perceived. Indeed, it is that very uncertainty that creates the opportunity.

1.2. Why Choose an Opportunity

Two questions

• why is there an opportunity in a particular location and moment (here and now) to introduce something novel to the marketplace?
• why are you the entrepreneur or founding team to translate this opportunity into a reality?

1.3. Value Creation and Capture: Why Here and Now

1. Photography

Photography means Writing with Lights

2. Elements

Record of people and things. Photo serves as document or symbol?

Making photo rather than Taking

Well organized/composited photo

Learning photography is just like learning language/writing

2.1. Shape

Syntax creates meaning, we arrange elements

Just 2D information

1. Why

思考路径

1. 减少计算量，将rgb中的运算空间缩减为bool的空间，大大减少计算
2. 一种想法就是在图像中筛选出固定颜色的物体
3. 筛选颜色可以使用RGB或者HSV，使用HSV可以解放亮度的影响，所以优先使用
4. 重点在于如何寻找HSV的合适范围筛选出想要的物体
5. 通过一张样片，进行测试，选取合适的上下界

2. Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

import cv2
import numpy as np

def nothing(x):
pass

# Load image
image = cv2.imread('1.jpg')

# Create a window
cv2.namedWindow('image')

# Create trackbars for color change
# Hue is from 0-179 for Opencv
cv2.createTrackbar('HMin', 'image', 140, 179, nothing)
cv2.createTrackbar('SMin', 'image', 0, 255, nothing)
cv2.createTrackbar('VMin', 'image', 0, 255, nothing)
cv2.createTrackbar('HMax', 'image', 160, 179, nothing)
cv2.createTrackbar('SMax', 'image', 0, 255, nothing)
cv2.createTrackbar('VMax', 'image', 0, 255, nothing)

# Set default value for Max HSV trackbars
cv2.setTrackbarPos('HMax', 'image', 179)
cv2.setTrackbarPos('SMax', 'image', 255)
cv2.setTrackbarPos('VMax', 'image', 255)

# Initialize HSV min/max values
hMin = sMin = vMin = hMax = sMax = vMax = 0
phMin = psMin = pvMin = phMax = psMax = pvMax = 0

while(1):
# Get current positions of all trackbars
hMin = cv2.getTrackbarPos('HMin', 'image')
sMin = cv2.getTrackbarPos('SMin', 'image')
vMin = cv2.getTrackbarPos('VMin', 'image')
hMax = cv2.getTrackbarPos('HMax', 'image')
sMax = cv2.getTrackbarPos('SMax', 'image')
vMax = cv2.getTrackbarPos('VMax', 'image')

# Set minimum and maximum HSV values to display
lower = np.array([hMin, sMin, vMin])
upper = np.array([hMax, sMax, vMax])

# Convert to HSV format and color threshold
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, lower, upper)
result = cv2.bitwise_and(image, image, mask=mask)

# Print if there is a change in HSV value
if((phMin != hMin) | (psMin != sMin) | (pvMin != vMin) | (phMax != hMax) | (psMax != sMax) | (pvMax != vMax) ):
print("(hMin = %d , sMin = %d, vMin = %d), (hMax = %d , sMax = %d, vMax = %d)" % (hMin , sMin , vMin, hMax, sMax , vMax))
phMin = hMin
psMin = sMin
pvMin = vMin
phMax = hMax
psMax = sMax
pvMax = vMax

# Display result image
cv2.imshow('image', result)
if cv2.waitKey(10) & 0xFF == ord('q'):
break
cv2.destroyAllWindows()

3. Ref

https://stackoverflow.com/questions/10948589/choosing-the-correct-upper-and-lower-hsv-boundaries-for-color-detection-withcv

1. 图片模糊

有多种原因会造成图片的模糊，主要有以下两种

• 运动模糊，因为在曝光时间内相机或者被摄对象运动造成
• 失焦模糊，因为对焦失败，被摄物体不在景深范围内造成

2. 运动模糊

首先考虑运动模糊

本质就是一个运动模糊核（Box filter）和清晰图像的卷积，频域上是一个sinc函数

难点

• 卷积核难以获取，和远近、运动速度和方向都有关
• 背景分割
• 卷积核丢失了高频信息，恢复之后信噪比低

做一些假设，降低问题的难度