本篇对于Python操作MySQL主要使用两种方式:
- 原生模块 pymsql
- ORM框架 SQLAchemy
pymysql
pymsql是Python中操作MySQL的模块,其使用方法和MySQLdb几乎相同。
下载安装
1
|
pip3 install pymysql |
使用操作
1、执行SQL
1 import pymysql 2 3 # 创建连接 4 conn = pymysql.connect(host='127.0.0.1', port=3306, user='root', passwd='123', db='t1') 5 # 创建游标 6 cursor = conn.cursor() 7 8 # 执行SQL,并返回收影响行数 9 effect_row = cursor.execute("update hosts set host = '1.1.1.2'") 10 11 # 执行SQL,并返回受影响行数 12 #effect_row = cursor.execute("update hosts set host = '1.1.1.2' where nid > %s", (1,)) 13 14 # 执行SQL,并返回受影响行数 15 #effect_row = cursor.executemany("insert into hosts(host,color_id)values(%s,%s)", [("1.1.1.11",1),("1.1.1.11",2)]) 16 17 18 # 提交,不然无法保存新建或者修改的数据 19 conn.commit() 20 21 # 关闭游标 22 cursor.close() 23 # 关闭连接 24 conn.close()
2、获取新创建数据自增ID
1 import pymysql 2 3 conn = pymysql.connect(host='127.0.0.1', port=3306, user='root', passwd='123', db='t1') 4 cursor = conn.cursor() 5 cursor.executemany("insert into hosts(host,color_id)values(%s,%s)", [("1.1.1.11",1),("1.1.1.11",2)]) 6 conn.commit() 7 cursor.close() 8 conn.close() 9 10 # 获取最新自增ID 11 new_id = cursor.lastrowid
3、获取查询数据
1 import pymysql 2 3 conn = pymysql.connect(host='127.0.0.1', port=3306, user='root', passwd='123', db='t1') 4 cursor = conn.cursor() 5 cursor.execute("select * from hosts") 6 7 # 获取第一行数据 8 row_1 = cursor.fetchone() 9 10 # 获取前n行数据 11 # row_2 = cursor.fetchmany(3) 12 # 获取所有数据 13 # row_3 = cursor.fetchall() 14 15 conn.commit() 16 cursor.close() 17 conn.close()
注:在fetch数据时按照顺序进行,可以使用cursor.scroll(num,mode)来移动游标位置,如:
- cursor.scroll(1,mode='relative') # 相对当前位置移动
- cursor.scroll(2,mode='absolute') # 相对绝对位置移动
4、fetch数据类型
关于默认获取的数据是元祖类型,如果想要或者字典类型的数据,即:
1 import pymysql 2 3 conn = pymysql.connect(host='127.0.0.1', port=3306, user='root', passwd='123', db='t1') 4 5 # 游标设置为字典类型 6 cursor = conn.cursor(cursor=pymysql.cursors.DictCursor) 7 r = cursor.execute("call p1()") 8 9 result = cursor.fetchone() 10 11 conn.commit() 12 cursor.close() 13 conn.close()
ORM框架 sqlalchemy
1.ORM介绍
orm英文全称object relational mapping,就是对象映射关系程序,简单来说我们类似python这种面向对象的程序来说一切皆对象,但是我们使用的数据库却都是关系型的,为了保证一致的使用习惯,通过orm将编程语言的对象模型和数据库的关系模型建立映射关系,这样我们在使用编程语言对数据库进行操作的时候可以直接使用编程语言的对象模型进行操作就可以了,而不用直接使用sql语言。
orm的优点:
- 隐藏了数据访问细节,“封闭”的通用数据库交互,ORM的核心。他使得我们的通用数据库交互变得简单易行,并且完全不用考虑该死的SQL语句。快速开发,由此而来。
- ORM使我们构造固化数据结构变得简单易行。
缺点:
无可避免的,自动化意味着映射和关联管理,代价是牺牲性能(早期,这是所有不喜欢ORM人的共同点)。现在的各种ORM框架都在尝试使用各种方法来减轻这块(LazyLoad,Cache),效果还是很显著的。
2.sqlalchemy安装
在Python中,最有名的ORM框架是SQLAlchemy。用户包括openstack\Dropbox等知名公司或应用,主要用户列表http://www.sqlalchemy.org/organizations.html#openstack
Dialect用于和数据API进行交流,根据配置文件的不同调用不同的数据库API,从而实现对数据库的操作,如:
1
2
3
4
5
6
7
8
9
10
11
12
13
|
MySQL-Python mysql+mysqldb: // <user>:<password>@<host>[:<port>]/<dbname> pymysql mysql+pymysql: // <username>:<password>@<host>/<dbname>[?<options>] MySQL-Connector mysql+mysqlconnector: // <user>:<password>@<host>[:<port>]/<dbname> cx_Oracle oracle+cx_oracle: //user :pass@host:port /dbname [?key=value&key=value...] 更多详见:http: //docs .sqlalchemy.org /en/latest/dialects/index .html |
安装sqlalchemy
1
|
pip install SQLAlchemy<br><br>pip install pymysql #由于mysqldb依然不支持py3,所以这里我们用pymysql与sqlalchemy交互 |
3.sqlalchemy基本使用
创建表
import sqlalchemy from sqlalchemy import create_engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import Column, Integer, String # 创建实例,并连接test库 engine = create_engine("mysql+pymysql://root:123456@localhost/test", encoding='utf-8', echo=True) # echo=True 显示信息 Base = declarative_base() # 生成orm基类 class User(Base): __tablename__ = 'user' # 表名 id = Column(Integer, primary_key=True) name = Column(String(32)) password = Column(String(64)) Base.metadata.create_all(engine) #创建表结构 (这里是父类调子类)
运行,显示相关信息,包括生成的sql语句:
CREATE TABLE user ( id INTEGER NOT NULL AUTO_INCREMENT, name VARCHAR(32), password VARCHAR(64), PRIMARY KEY (id) )
除上面的创建之外,还有一种创建表的方式
1 from sqlalchemy import Table, MetaData, Column, Integer, String, ForeignKey 2 from sqlalchemy.orm import mapper 3 4 metadata = MetaData() 5 6 user = Table('user', metadata, 7 Column('id', Integer, primary_key=True), 8 Column('name', String(50)), 9 Column('fullname', String(50)), 10 Column('password', String(12)) 11 ) 12 13 class User(object): 14 def __init__(self, name, fullname, password): 15 self.name = name 16 self.fullname = fullname 17 self.password = password 18 19 mapper(User, user) # 类User 和 user关联起来 20 # the table metadata is created separately with the Table construct, 21 # then associated with the User class via the mapper() function 22 # 如果数据库里有,就不会创建了。
事实上,我们用第一种方式创建的表就是基于第2种方式的再封装。
插入数据
from sqlalchemy import create_engine from sqlalchemy import Table, MetaData, Column, Integer, String from sqlalchemy.orm import mapper, sessionmaker # 创建实例,并连接test库 engine = create_engine("mysql+pymysql://root:123456@localhost/test", encoding='utf-8', echo=True) metadata = MetaData() user = Table('user', metadata, Column('id', Integer, primary_key=True), Column('name', String(50)), Column('password', String(12)) ) class User(object): def __init__(self, name, id, password): self.id = id self.name = name self.password = password # the table metadata is created separately with the Table construct, then associated with the User class via the mapper() function mapper(User, user) # 创建与数据库的会话session class ,注意,这里返回给session的是个class,不是实例 Session_class = sessionmaker(bind=engine) # 实例和engine绑定 Session = Session_class() # 生成session实例,相当于游标 user_obj = User(id=27,name="fgf",password="123456") # 生成你要创建的数据对象 print(user_obj.name,user_obj.id) # 此时还没创建对象呢,不信你打印一下id发现还是None Session.add(user_obj) # 把要创建的数据对象添加到这个session里, 一会统一创建 print(user_obj.name,user_obj.id) #此时也依然还没创建 Session.commit() #现此才统一提交,创建数据
查询数据
1
2
|
my_user = Session.query(User).filter_by(name = "alex" ).first() print (my_user) |
此时你看到的输出是这样的应该
1
|
<__main__.User object at 0x105b4ba90 > |
我擦,这是什么?这就是你要的数据呀, 只不过sqlalchemy帮你把返回的数据映射成一个对象啦,这样你调用每个字段就可以跟调用对象属性一样啦,like this..
1
2
3
4
|
print (my_user. id ,my_user.name,my_user.password) 输出 1 alex alex3714 |
不过刚才上面的显示的内存对象对址你是没办法分清返回的是什么数据的,除非打印具体字段看一下,如果想让它变的可读,只需在定义表的类下面加上这样的代码
1 def __repr__(self): 2 return "<User(name='%s', password='%s')>" % ( 3 self.name, self.password) 4 5 from sqlalchemy import create_engine 6 from sqlalchemy import Table, MetaData, Column, Integer, String 7 from sqlalchemy.orm import mapper, sessionmaker 8 # 创建实例,并连接test库 9 engine = create_engine("mysql+pymysql://root:123456@localhost/test", 10 encoding='utf-8', echo=True) 11 12 metadata = MetaData() 13 14 user = Table('user', metadata, 15 Column('id', Integer, primary_key=True), 16 Column('name', String(50)), 17 Column('password', String(12)) 18 ) 19 class User(object): 20 def __init__(self, name, id, password): 21 self.id = id 22 self.name = name 23 self.password = password 24 def __repr__(self): 25 return "<User(name='%s', password='%s')>" % (self.name, self.password) 26 27 mapper(User, user) 28 # 创建与数据库的会话session class ,注意,这里返回给session的是个class,不是实例 29 Session_class = sessionmaker(bind=engine) 30 Session = Session_class() # 生成session实例 31 32 my_user = Session.query(User).filter_by(name="fgf").first() # 查询第一个 33 # my_user = Session.query(User).filter_by().all() # 查询所有 34 print(my_user) 35 # print(my_user.id,my_user.name,my_user.password) 36 37 # Session.commit() #查询不需要commit
获取所有数据
print(Session.query(User.name,User.id).all() )
多条件查询
filter_by与filter
my_user1 = Session.query(User).filter(User.id>2).all() my_user2 = Session.query(User).filter_by(id=27).all() # filter_by相等用‘=’ my_user3 = Session.query(User).filter(User.id==27).all() # filter相等用‘==’ print(my_user1,' ',my_user2,' ',my_user3)
多条件查询
objs = Session.query(User).filter(User.id>0).filter(User.id<7).all()
上面2个filter的关系相当于 user.id >1 AND user.id <7 的效果
修改
1
2
3
4
5
|
my_user = Session.query(User).filter_by(name = "alex" ).first() my_user.name = "Alex Li" Session.commit() |
回滚
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
my_user = Session.query(User).filter_by( id = 1 ).first() my_user.name = "Jack" fake_user = User(name = 'Rain' , password = '12345' ) Session.add(fake_user) print (Session.query(User). filter (User.name.in_([ 'Jack' , 'rain' ])). all () ) #这时看session里有你刚添加和修改的数据 Session.rollback() #此时你rollback一下 print (Session.query(User). filter (User.name.in_([ 'Jack' , 'rain' ])). all () ) #再查就发现刚才添加的数据没有了。 # Session # Session.commit() |
获取所有数据
1
|
print (Session.query(User.name,User. id ). all () ) |
多条件查询
1
|
objs = Session.query(User). filter (User. id > 0 ). filter (User. id < 7 ). all () |
上面2个filter的关系相当于 user.id >1 AND user.id <7 的效果
统计和分组
1
|
Session.query(User). filter (User.name.like( "Ra%" )).count() |
分组
1
2
|
from sqlalchemy import func print (Session.query(func.count(User.name),User.name).group_by(User.name). all () ) |
相当于原生sql为
1
2
|
SELECT count(user.name) AS count_1, user.name AS user_name FROM user GROUP BY user.name |
输出为
[(1, 'Jack'), (2, 'Rain')]
外键关联
准备工作:先创建一个表,再插入数据
from sqlalchemy import create_engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import Column, Integer, String from sqlalchemy.orm import sessionmaker # 创建实例,并连接test库 engine = create_engine("mysql+pymysql://root:123456@localhost/test", encoding='utf-8', echo=True) Base = declarative_base() # 生成orm基类 class Student(Base): __tablename__ = 'student' # 表名 id = Column(Integer, primary_key=True, autoincrement=True) stu_id = Column(Integer) age = Column(Integer) # 整型 gender = Column(Enum('M','F'),nullable=False) Base.metadata.create_all(engine) #创建表结构 (这里是父类调子类) # 创建与数据库的会话session class ,注意,这里返回给session的是个class,不是实例 Session_class = sessionmaker(bind=engine) Session = Session_class() # 生成session实例 stu_obj = Student(stu_id=27, age=22, gender="M") Session.add(stu_obj) Session.commit() #现此才统一提交,创建数据
连表
ret = session.query(Users, Favor).filter(Users.id == Favor.nid).all() # 以下两种 必须表之间有外键关联才能查 ret = session.query(Person).join(Favor).all() ret = session.query(Person).join(Favor, isouter=True).all()
第一种示例:
from sqlalchemy import create_engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import Column, Integer, String, Enum from sqlalchemy.orm import sessionmaker # 创建实例,并连接test库 engine = create_engine("mysql+pymysql://root:123456@localhost/test", encoding='utf-8') Base = declarative_base() # 生成orm基类 class User(Base): __tablename__ = 'user' # 表名 id = Column(Integer, primary_key=True) name = Column(String(32)) password = Column(String(64)) def __repr__(self): return "[%s name:%s]" %(self.id, self.name) class Student(Base): __tablename__ = 'student' # 表名 id = Column(Integer, primary_key=True, autoincrement=True) stu_id = Column(Integer) age = Column(Integer) # 整型 gender = Column(Enum('M','F'),nullable=False) def __repr__(self): return "[%s stu_id:%s sex:%s]" %(self.stu_id, self.age, self.gender) Session_class = sessionmaker(bind=engine) Session = Session_class() # 生成session实例 res = Session.query(User, Student).filter(User.id == Student.stu_id).all() print(res)
外键关联实现
from sqlalchemy import create_engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import Column, Enum,DATE,Integer, String,ForeignKey from sqlalchemy.orm import sessionmaker,relationship engine = create_engine("mysql+pymysql://root:123456@localhost/test", encoding='utf-8') Base = declarative_base() # 生成orm基类 class Stu2(Base): __tablename__ = "stu2" id = Column(Integer, primary_key=True) name = Column(String(32),nullable=False) register_date = Column(DATE,nullable=False) def __repr__(self): return "<%s name:%s>" % (self.id, self.name) class StudyRecord(Base): __tablename__ = "study_record" id = Column(Integer, primary_key=True) day = Column(Integer,nullable=False) status = Column(String(32),nullable=False) stu_id = Column(Integer,ForeignKey("stu2.id")) #------外键关联------ #这个nb,允许你在user表里通过backref字段反向查出所有它在stu2表里的关联项数据 stu2 = relationship("Stu2", backref="my_study_record") # 添加关系,反查(在内存里) def __repr__(self): return "<%s day:%s status:%s>" % (self.stu2.name, self.day,self.status) Base.metadata.create_all(engine) # 创建表结构 Session_class = sessionmaker(bind=engine) # 创建与数据库的会话session class ,注意,这里返回给session的是个class,不是实例 session = Session_class() # 生成session实例 #cursor s1 = Stu2(name="A",register_date="2014-05-21") s2 = Stu2(name="J",register_date="2014-03-21") s3 = Stu2(name="R",register_date="2014-02-21") s4 = Stu2(name="E",register_date="2013-01-21") study_obj1 = StudyRecord(day=1,status="YES", stu_id=1) study_obj2 = StudyRecord(day=2,status="NO", stu_id=1) study_obj3 = StudyRecord(day=3,status="YES", stu_id=1) study_obj4 = StudyRecord(day=1,status="YES", stu_id=2) session.add_all([s1,s2,s3,s4,study_obj1,study_obj2,study_obj3,study_obj4]) # 创建 session.commit() stu_obj = session.query(Stu2).filter(Stu2.name=="a").first() # 查询 # 在stu2表,查到StudyRecord表的记录 print(stu_obj.my_study_record) # 查询A一共上了几节课
可以查看创建命令:
show create table study_record;
多外键关联
多外键关联,并且关联同一个表。
下表中,Customer表有2个字段都关联了Address表
cat orm_many_fk.py
from sqlalchemy import Integer, ForeignKey, String, Column from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import relationship from sqlalchemy import create_engine Base = declarative_base() class Customer(Base): __tablename__ = 'customer' id = Column(Integer, primary_key=True) name = Column(String(64)) # 账单地址和邮寄地址 都关联同一个地址表 billing_address_id = Column(Integer, ForeignKey("address.id")) shipping_address_id = Column(Integer, ForeignKey("address.id")) billing_address = relationship("Address", foreign_keys=[billing_address_id]) shipping_address = relationship("Address", foreign_keys=[shipping_address_id]) class Address(Base): __tablename__ = 'address' id = Column(Integer, primary_key=True) street = Column(String(64)) city = Column(String(64)) state = Column(String(64)) def __repr__(self): return self.street engine = create_engine("mysql+pymysql://root:123456@localhost/test", encoding='utf-8') Base.metadata.create_all(engine) # 创建表结构
MariaDB [test]> show create table customer; | customer | CREATE TABLE `customer` ( `id` int(11) NOT NULL AUTO_INCREMENT, `name` varchar(64) DEFAULT NULL, `billing_address_id` int(11) DEFAULT NULL, `shipping_address_id` int(11) DEFAULT NULL, PRIMARY KEY (`id`), KEY `billing_address_id` (`billing_address_id`), KEY `shipping_address_id` (`shipping_address_id`), CONSTRAINT `customer_ibfk_1` FOREIGN KEY (`billing_address_id`) REFERENCES `address` (`id`), CONSTRAINT `customer_ibfk_2` FOREIGN KEY (`shipping_address_id`) REFERENCES `address` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 | MariaDB [test]> desc customer; +---------------------+-------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +---------------------+-------------+------+-----+---------+----------------+ | id | int(11) | NO | PRI | NULL | auto_increment | | name | varchar(64) | YES | | NULL | | | billing_address_id | int(11) | YES | MUL | NULL | | | shipping_address_id | int(11) | YES | MUL | NULL | | +---------------------+-------------+------+-----+---------+----------------+
正常写的时候,表结构单独写一个模块。添加数据
import orm_many_fk from sqlalchemy.orm import sessionmaker Session_class = sessionmaker(bind=orm_many_fk.engine) # 创建与数据库的会话session class ,注意,这里返回给session的是个class,不是实例 session = Session_class() # 生成session实例 #cursor addr1 = orm_many_fk.Address(street="Tiantongyuan", city="ChangPing", state="BJ") addr2 = orm_many_fk.Address(street="Wudaokou", city="Haidian", state="BJ") addr3 = orm_many_fk.Address(street="Yanjiao", city="LangFang", state="HB") session.add_all([addr1,addr2,addr3]) c1 = orm_many_fk.Customer(name="Fgf", billing_address= addr1,shipping_address=addr2) c2 = orm_many_fk.Customer(name="Jack", billing_address= addr3,shipping_address=addr3) session.add_all([c1,c2]) session.commit() obj = session.query(orm_many_fk.Customer).filter(orm_many_fk.Customer.name=="Fgf").first() print(obj.name,obj.billing_address,obj.shipping_address) # 查询
多对多关联
现在来设计一个能描述“图书”与“作者”的关系的表结构,需求是
- 一本书可以有好几个作者一起出版
- 一个作者可以写好几本书
此时你会发现,用之前学的外键好像没办法实现上面的需求了
那怎么办呢? 此时,我们可以再搞出一张中间表,就可以了
这样就相当于通过book_m2m_author表完成了book表和author表之前的多对多关联
双向一对多,就是多对多。
用orm如何表示呢?
# 创建表结构 from sqlalchemy import Table, Column, Integer,String,DATE, ForeignKey from sqlalchemy.orm import relationship from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import create_engine from sqlalchemy.orm import sessionmaker Base = declarative_base() # 第三张表 自己创建。不需要手动管理,orm自动维护 book_m2m_author = Table('book_m2m_author', Base.metadata, Column('book_id',Integer,ForeignKey('books.id')), Column('author_id',Integer,ForeignKey('authors.id')), ) class Book(Base): __tablename__ = 'books' id = Column(Integer,primary_key=True) name = Column(String(64)) pub_date = Column(DATE) # book表不知道第三张表,所以关联一下第三张表 authors = relationship('Author',secondary=book_m2m_author,backref='books') def __repr__(self): return self.name class Author(Base): __tablename__ = 'authors' id = Column(Integer, primary_key=True) name = Column(String(32)) def __repr__(self): return self.name engine = create_engine("mysql+pymysql://root:123456@localhost/test", encoding='utf-8') Base.metadata.create_all(engine) # 创建表结构
MariaDB [test]> desc book_m2m_author; +-----------+---------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------+---------+------+-----+---------+-------+ | book_id | int(11) | YES | MUL | NULL | | | author_id | int(11) | YES | MUL | NULL | | +-----------+---------+------+-----+---------+-------+ # 做了复合键了
# 添加数据 import orm_m2m from sqlalchemy.orm import sessionmaker Session_class = sessionmaker(bind=orm_m2m.engine) # 创建与数据库的会话session class ,注意,这里返回给session的是个class,不是实例 session = Session_class() # 生成session实例 #cursor # 创建书 b1 = orm_m2m.Book(name="learn python with Alex",pub_date="2014-05-2") b2= orm_m2m.Book(name="learn Zhangbility with Alex",pub_date="2015-05-2") b3 = orm_m2m.Book(name="Learn hook up girls with Alex",pub_date="2016-05-2") # 创建作者 a1 = orm_m2m.Author(name="Alex") a2 = orm_m2m.Author(name="Jack") a3 = orm_m2m.Author(name="Rain") # 关联关系 b1.authors = [a1,a3] b3.authors = [a1,a2,a3] session.add_all([b1,b2,b3,a1,a2,a3]) session.commit()
# 重要是查询 author_obj = session.query(orm_m2m.Author).filter(orm_m2m.Author.name=="alex").first() print(author_obj.books[0:]) book_obj = session.query(orm_m2m.Book).filter(orm_m2m.Book.id==2).first() print(book_obj.authors)
多对多删除
删除数据时不用管boo_m2m_authors , sqlalchemy会自动帮你把对应的数据删除
通过书删除作者
author_obj =s.query(Author).filter_by(name="Jack").first() book_obj = s.query(Book).filter_by(name="跟Alex学把妹").first() book_obj.authors.remove(author_obj) #从一本书里删除一个作者 s.commit()
直接删除作者
删除作者时,会把这个作者跟所有书的关联关系数据也自动删除
author_obj =s.query(Author).filter_by(name="Alex").first() # print(author_obj.name , author_obj.books) s.delete(author_obj) s.commit()
4.中文的问题
先查看数据库的字符集
MariaDB [test]> show create database test;
+----------+-----------------------------------------------------------------+
| Database | Create Database |
+----------+-----------------------------------------------------------------+
| test | CREATE DATABASE `test` /*!40100 DEFAULT CHARACTER SET latin1 */ |
+----------+-----------------------------------------------------------------+
修改数据库的字符集(如果修改字符集,添加仍不显示中文,可能就需要创建是指定尝试一下了。)
MariaDB [test]> alter database test character set utf8;
创建数据库指定数据库的字符集
MariaDB [test]> create database mydb character set utf8;
sqlalchemy 连接指定
engine = create_engine("mysql+pymysql://root:123456@localhost/test?charset=utf8",)