如何解决Django中跨数据库的外键缺乏支持

How to work around lack of support for foreign keys across databases in Django

我知道Django不支持跨多个数据库的外键(最初是Django 1.3 docs)

但我正在寻找一种解决方法。

什么行不通

我在一个单独的数据库上有两个模型。

routers.py：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

fruit_app / models.py中的模型1：

1
2
3
4

from django.db import models

class Fruit(models.Model):
name = models.CharField(max_length=20)

news_app / models.py中的模型2：

1
2
3
4
5

from django.db import models

class Article(models.Model):
fruit = models.ForeignKey('fruit_app.Fruit')
intro = models.TextField()

尝试在管理员中添加"文章"会出现以下错误，因为它在错误的数据库('news_db')上查找Fruit模型：

1
2
3

DatabaseError at /admin/news_app/article/add/

(1146,"Table 'fkad_news.fruit_app_fruit' doesn't exist")

方法1：子类IntegerField

我创建了一个自定义字段ForeignKeyAcrossDb，它是IntegerField的子类。代码在github上：https：//github.com/saltycrane/django-foreign-key-across-db-testproject/tree/integerfield_subclass

fields.py：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

from django.db import models

class ForeignKeyAcrossDb(models.IntegerField):
'''
Exists because foreign keys do not work across databases
'''
def __init__(self, model_on_other_db, **kwargs):
self.model_on_other_db = model_on_other_db
super(ForeignKeyAcrossDb, self).__init__(**kwargs)

def to_python(self, value):
# TODO: this db lookup is duplicated in get_prep_lookup()
if isinstance(value, self.model_on_other_db):
return value
else:
return self.model_on_other_db._default_manager.get(pk=value)

def get_prep_value(self, value):
if isinstance(value, self.model_on_other_db):
value = value.pk
return super(ForeignKeyAcrossDb, self).get_prep_value(value)

def get_prep_lookup(self, lookup_type, value):
# TODO: this db lookup is duplicated in to_python()
if not isinstance(value, self.model_on_other_db):
value = self.model_on_other_db._default_manager.get(pk=value)

return super(ForeignKeyAcrossDb, self).get_prep_lookup(lookup_type, value)

我将我的文章模型更改为：

1
2
3

class Article(models.Model):
fruit = ForeignKeyAcrossDb(Fruit)
intro = models.TextField()

问题是，有时当我访问Article.fruit时，它是一个整数，有时它是Fruit对象。我希望它永远是一个Fruit对象。我需要做什么来访问Article.fruit总是返回一个Fruit对象？

作为我的解决方法的解决方法，我添加了一个fruit_obj属性，但如果可能的话我想删除它：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

class Article(models.Model):
fruit = ForeignKeyAcrossDb(Fruit)
intro = models.TextField()

# TODO: shouldn't need fruit_obj if ForeignKeyAcrossDb field worked properly
@property
def fruit_obj(self):
if not hasattr(self, '_fruit_obj'):
# TODO: why is it sometimes an int and sometimes a Fruit object?
if isinstance(self.fruit, int) or isinstance(self.fruit, long):
print 'self.fruit IS a number'
self._fruit_obj = Fruit.objects.get(pk=self.fruit)
else:
print 'self.fruit IS NOT a number'
self._fruit_obj = self.fruit
return self._fruit_obj

def fruit_name(self):
return self.fruit_obj.name

方法2：子类ForeignKey字段

作为第二次尝试，我尝试了对ForeignKey字段进行子类化。我修改了ReverseSingleRelatedObjectDescriptor以在Fruit的模型管理器上使用forced_using指定的数据库。我还删除了ForeignKey子类上的validate()方法。此方法与方法1没有相同的问题。在github上的代码：https：//github.com/saltycrane/django-foreign-key-across-db-testproject/tree/foreignkey_subclass

fields.py：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

from django.db import models
from django.db import router
from django.db.models.query import QuerySet

class ReverseSingleRelatedObjectDescriptor(object):
# This class provides the functionality that makes the related-object
# managers available as attributes on a model class, for fields that have
# a single"remote" value, on the class that defines the related field.
# In the example"choice.poll", the poll attribute is a
# ReverseSingleRelatedObjectDescriptor instance.
def __init__(self, field_with_rel):
self.field = field_with_rel

def __get__(self, instance, instance_type=None):
if instance is None:
return self

cache_name = self.field.get_cache_name()
try:
return getattr(instance, cache_name)
except AttributeError:
val = getattr(instance, self.field.attname)
if val is None:
# If NULL is an allowed value, return it.
if self.field.null:
return None
raise self.field.rel.to.DoesNotExist
other_field = self.field.rel.get_related_field()
if other_field.rel:
params = {'%s__pk' % self.field.rel.field_name: val}
else:
params = {'%s__exact' % self.field.rel.field_name: val}

# If the related manager indicates that it should be used for
# related fields, respect that.
rel_mgr = self.field.rel.to._default_manager
db = router.db_for_read(self.field.rel.to, instance=instance)
if getattr(rel_mgr, 'forced_using', False):
db = rel_mgr.forced_using
rel_obj = rel_mgr.using(db).get(**params)
elif getattr(rel_mgr, 'use_for_related_fields', False):
rel_obj = rel_mgr.using(db).get(**params)
else:
rel_obj = QuerySet(self.field.rel.to).using(db).get(**params)
setattr(instance, cache_name, rel_obj)
return rel_obj

def __set__(self, instance, value):
raise NotImplementedError()

class ForeignKeyAcrossDb(models.ForeignKey):

def contribute_to_class(self, cls, name):
models.ForeignKey.contribute_to_class(self, cls, name)
setattr(cls, self.name, ReverseSingleRelatedObjectDescriptor(self))
if isinstance(self.rel.to, basestring):
target = self.rel.to
else:
target = self.rel.to._meta.db_table
cls._meta.duplicate_targets[self.column] = (target,"o2m")

def validate(self, value, model_instance):
pass

fruit_app / models.py：

1
2
3
4
5
6
7
8
9
10
11

from django.db import models

class FruitManager(models.Manager):
forced_using = 'default'

class Fruit(models.Model):
name = models.CharField(max_length=20)

objects = FruitManager()

news_app / models.py：

1
2
3
4
5
6
7
8
9
10
11
12

from django.db import models

from foreign_key_across_db_testproject.fields import ForeignKeyAcrossDb
from foreign_key_across_db_testproject.fruit_app.models import Fruit

class Article(models.Model):
fruit = ForeignKeyAcrossDb(Fruit)
intro = models.TextField()

def fruit_name(self):
return self.fruit.name

方法2a：为fruit_app添加路由器

此解决方案使用额外的路由器fruit_app。此解决方案不需要修改方法2中所需的ForeignKey。在查看Django在django.db.utils.ConnectionRouter中的默认路由行为之后，我们发现即使我们希望fruit_app默认位于'default'数据库上，传递给db_for_read的instance提示进行外键查找将其放在'news_db'数据库上。我们添加了第二个路由器，以确保始终从'default'数据库中读取fruit_app模型。 ForeignKey子类仅用于"修复"ForeignKey.validate()方法。 (如果Django希望跨数据库支持外键，我会说这是一个Django错误。)
代码在github上：https：//github.com/saltycrane/django-foreign-key-across-db-testproject

routers.py：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

class NewsRouter(object):
def db_for_read(self, model, **hints):
if model._meta.app_label == 'news_app':
return 'news_db'
return None

def db_for_write(self, model, **hints):
if model._meta.app_label == 'news_app':
return 'news_db'
return None

def allow_relation(self, obj1, obj2, **hints):
if obj1._meta.app_label == 'news_app' or obj2._meta.app_label == 'news_app':
return True
return None

def allow_syncdb(self, db, model):
if db == 'news_db':
return model._meta.app_label == 'news_app'
elif model._meta.app_label == 'news_app':
return False
return None

class FruitRouter(object):
def db_for_read(self, model, **hints):
if model._meta.app_label == 'fruit_app':
return 'default'
return None

def db_for_write(self, model, **hints):
if model._meta.app_label == 'fruit_app':
return 'default'
return None

def allow_relation(self, obj1, obj2, **hints):
if obj1._meta.app_label == 'fruit_app' or obj2._meta.app_label == 'fruit_app':
return True
return None

def allow_syncdb(self, db, model):
if db == 'default':
return model._meta.app_label == 'fruit_app'
elif model._meta.app_label == 'fruit_app':
return False
return None

fruit_app / models.py：

1
2
3
4
5

from django.db import models

class Fruit(models.Model):
name = models.CharField(max_length=20)

news_app / models.py：

1
2
3
4
5
6
7
8
9
10
11
12

fields.py：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

from django.core import exceptions
from django.db import models
from django.db import router

class ForeignKeyAcrossDb(models.ForeignKey):

def validate(self, value, model_instance):
if self.rel.parent_link:
return
models.Field.validate(self, value, model_instance)
if value is None:
return

using = router.db_for_read(self.rel.to, instance=model_instance) # is this more correct than Django's 1.2.5 version?
qs = self.rel.to._default_manager.using(using).filter(
**{self.rel.field_name: value}
)
qs = qs.complex_filter(self.rel.limit_choices_to)
if not qs.exists():
raise exceptions.ValidationError(self.error_messages['invalid'] % {
'model': self.rel.to._meta.verbose_name, 'pk': value})

附加信息

django-users列表中包含大量信息的主题：http：//groups.google.com/group/django-users/browse_thread/thread/74bcd1afdeb2f0/0fdfce061124b915
多数据库文档的修订历史记录：http：//code.djangoproject.com/log/django/trunk/docs/topics/db/multi-db.txt？verbose = on

更新

我们在调整了路由器之后实现了最后一种方法。整个实施过程非常痛苦，这使我们认为我们必须做错了。在TODO列表上正在为此编写单元测试。

相关讨论

几天后我的脑袋里，我设法让我的外键在同一个银行！

可以制作一个改变FORM来寻求一个不同银行的外键！

首先，添加一个FIELDS的RECHARGE，直接(破解)我的形式，在函数_init_

app.form.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

# -*- coding: utf-8 -*-
from django import forms
import datetime
from app_ti_helpdesk import models as mdp

#classe para formulario de Novo HelpDesk
class FormNewHelpDesk(forms.ModelForm):
class Meta:
model = mdp.TblHelpDesk
fields = (
"problema_alegado",
"cod_direcionacao",
"data_prevista",
"hora_prevista",
"atendimento_relacionado_a",
"status",
"cod_usuario",
)

def __init__(self, *args, **kwargs):
#-------------------------------------
# using remove of kwargs
#-------------------------------------
db = kwargs.pop("using", None)

# CASE use Unique Keys
self.Meta.model.db = db

super(FormNewHelpDesk, self).__init__(*args,**kwargs)

#-------------------------------------
# recreates the fields manually
from copy import deepcopy
self.fields = deepcopy( forms.fields_for_model( self.Meta.model, self.Meta.fields, using=db ) )
#
#-------------------------------------

#### follows the standard template customization, if necessary

self.fields['problema_alegado'].widget.attrs['rows'] = 3
self.fields['problema_alegado'].widget.attrs['cols'] = 22
self.fields['problema_alegado'].required = True
self.fields['problema_alegado'].error_messages={'required': 'Necessário informar o motivo da solicita??o de ajuda!'}

self.fields['data_prevista'].widget.attrs['class'] = 'calendario'
self.fields['data_prevista'].initial = (datetime.timedelta(4)+datetime.datetime.now().date()).strftime("%Y-%m-%d")

self.fields['hora_prevista'].widget.attrs['class'] = 'hora'
self.fields['hora_prevista'].initial =datetime.datetime.now().time().strftime("%H:%M")

self.fields['status'].initial = '0' #aberto
self.fields['status'].widget.attrs['disabled'] = True

self.fields['atendimento_relacionado_a'].initial = '07'

self.fields['cod_direcionacao'].required = True
self.fields['cod_direcionacao'].label ="Direcionado a"
self.fields['cod_direcionacao'].initial = '2'
self.fields['cod_direcionacao'].error_messages={'required': 'Necessário informar para quem é direcionado a ajuda!'}

self.fields['cod_usuario'].widget = forms.HiddenInput()

从视图中调用表单

app.view.py

1	form = forms.FormNewHelpDesk(request.POST or None, using=banco)

现在，源代码DJANGO的变化

只有ForeignKey，ManyToManyField和OneToOneField类型的字段才能使用'using'，所以添加了一个IF ...

django.forms.models.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

# line - 133: add using=None
def fields_for_model(model, fields=None, exclude=None, widgets=None, formfield_callback=None, using=None):

# line - 159

if formfield_callback is None:
#----------------------------------------------------
from django.db.models.fields.related import (ForeignKey, ManyToManyField, OneToOneField)
if type(f) in (ForeignKey, ManyToManyField, OneToOneField):
kwargs['using'] = using

formfield = f.formfield(**kwargs)
#----------------------------------------------------
elif not callable(formfield_callback):
raise TypeError('formfield_callback must be a function or callable')
else:
formfield = formfield_callback(f, **kwargs)

更改关注文件

django.db.models.base.py

改变

1 2	# line 717 qs = model_class._default_manager.filter(**lookup_kwargs)

对于

1 2	# line 717 qs = model_class._default_manager.using(getattr(self, 'db', None)).filter(**lookup_kwargs)

准备好了：D

对于ForeignKeyAcrossDb部分，您是否可能在__init__内对您的班级进行一些调整？检查相应字段是否为Integer(如果不是)，从数据库加载，或执行任何其他必需操作。 Python __class__ es可以在运行时更改，没有太多问题。

我知道Djano-nosql支持密钥，虽然来自http://www.allbuttonspressed.com/projects/django-dbindexer的一些魔力。也许其中一些可能有所帮助。

从描述：

"你可以告诉dbindexer哪些模型和字段应该支持这些查询，它将负责为你维护所需的索引。"

-Kerry

您可以在数据库中创建一个包含跨数据库查询的视图，然后在单独的文件中定义视图的模型以使syncdb保持工作。

快乐的编程。 :)

相关讨论

此解决方案最初是为一个托管数据库编写的，其中包含迁移和一个或多个遗留数据库，其中模型Meta managed=False在数据库级别连接到同一数据库。如果db_table选项包含数据库名称加上由''(MySQL)或'''(其他数据库)正确引用的表名，例如db_table = '"DB2"."table_b"'，则Django不再引用它。查询由以下内容编译： Django ORM正确，即使使用JOIN：

1
2
3
4
5
6

class TableB(models.Model):
....
class Meta:
db_table = '`DB2`.`table_b`' # for MySQL
# db_table = '"DB2"."table_b"' # for all other backends
managed = False

查询集：

1
2
3

>>> qs = TableB.objects.all()
>>> str(qs.query)
'SELECT"DB2"."table_b"."id" FROM DB2"."table_b"'

这是由Django中的所有db后端支持的。

(似乎我在一个重复的新问题上开始了我的答案，我的答案仍在继续。)

遇到类似的需要跨多个(5)数据库引用(主要)静态数据的问题。对ReversedSingleRelatedObjectDescriptor稍作更新，以允许设置相关模型。它没有实现反向关系atm。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

class ReverseSingleRelatedObjectDescriptor(object):
"""
This class provides the functionality that makes the related-object managers available as attributes on a model
class, for fields that have a single"remote" value, on the class that defines the related field. Used with
LinkedField.
"""
def __init__(self, field_with_rel):
self.field = field_with_rel
self.cache_name = self.field.get_cache_name()

def __get__(self, instance, instance_type=None):
if instance is None:
return self

try:
return getattr(instance, self.cache_name)
except AttributeError:
val = getattr(instance, self.field.attname)
if val is None:
# If NULL is an allowed value, return it
if self.field.null:
return None
raise self.field.rel.to.DoesNotExist
other_field = self.field.rel.get_related_field()
if other_field.rel:
params = {'%s__pk' % self.field.rel.field_name: val}
else:
params = {'%s__exact' % self.field.rel.field_name: val}

# If the related manager indicates that it should be used for related fields, respect that.
rel_mgr = self.field.rel.to._default_manager
db = router.db_for_read(self.field.rel.to, instance=instance)
if getattr(rel_mgr, 'forced_using', False):
db = rel_mgr.forced_using
rel_obj = rel_mgr.using(db).get(**params)
elif getattr(rel_mgr, 'use_for_related_fields', False):
rel_obj = rel_mgr.using(db).get(**params)
else:
rel_obj = QuerySet(self.field.rel.to).using(db).get(**params)
setattr(instance, self.cache_name, rel_obj)
return rel_obj

def __set__(self, instance, value):
if instance is None:
raise AttributeError("%s must be accessed via instance" % self.field.name)

# If null=True, we can assign null here, but otherwise the value needs to be an instance of the related class.
if value is None and self.field.null is False:
raise ValueError('Cannot assign None:"%s.%s" does not allow null values.' %
(instance._meta.object_name, self.field.names))
elif value is not None and not isinstance(value, self.field.rel.to):
raise ValueError('Cannot assign"%r":"%s.%s" must be a"%s" instance.' %
(value, instance._meta.object_name, self.field.name, self.field.rel.to._meta.object_name))
elif value is not None:
# Only check the instance state db, LinkedField implies that the value is on a different database
if instance._state.db is None:
instance._state.db = router.db_for_write(instance.__class__, instance=value)

# Is not used by OneToOneField, no extra measures to take here

# Set the value of the related field
try:
val = getattr(value, self.field.rel.get_related_field().attname)
except AttributeError:
val = None
setattr(instance, self.field.attname, val)

# Since we already know what the related object is, seed the related object caches now, too. This avoids another
# db hit if you get the object you just set
setattr(instance, self.cache_name, value)
if value is not None and not self.field.rel.multiple:
setattr(value, self.field.related.get_cache_name(), instance)

和

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

class LinkedField(models.ForeignKey):
"""
Field class used to link models across databases. Does not ensure referrential integraty like ForeignKey
"""
def _description(self):
return"Linked Field (type determined by related field)"

def contribute_to_class(self, cls, name):
models.ForeignKey.contribute_to_class(self, cls, name)
setattr(cls, self.name, ReverseSingleRelatedObjectDescriptor(self))
if isinstance(self.rel.to, basestring):
target = self.rel.to
else:
target = self.rel.to._meta.db_table
cls._meta.duplicate_targets[self.column] = (target,"o2m")

def validate(self, value, model_instance):
pass

外键字段暗示您可以
- 通过加入ie fruit__name查询关系
- 检查参照完整性
- 确保删除时的引用完整性
- 管理员原始ID查找功能
- (多一点...)

第一个用例总是有问题的。
可能在代码库中还有一些其他外键特殊情况也行不通。

我运行一个相当大的django网站，我们目前正在使用普通的整数字段。
现在我认为继承整数字段并将id添加到对象转换将是最简单的(在1.2中需要修补一些django，希望现在改进)
会让你知道我们找到了什么解决方案。

我有一个django v1.10的新解决方案。有两个部分。它适用于django.admin和django.rest-framework。

继承ForeignKey类并创建ForeignKeyAcrossDb，并根据此票证和此帖子覆盖validate()函数。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

class ForeignKeyAcrossDb(models.ForeignKey):
def validate(self, value, model_instance):
if self.remote_field.parent_link:
return
super(models.ForeignKey, self).validate(value, model_instance)
if value is None:
return
using = router.db_for_read(self.remote_field.model, instance=model_instance)
qs = self.remote_field.model._default_manager.using(using).filter(
**{self.remote_field.field_name: value}
)
qs = qs.complex_filter(self.get_limit_choices_to())
if not qs.exists():
raise exceptions.ValidationError(
self.error_messages['invalid'],
code='invalid',
params={
'model': self.remote_field.model._meta.verbose_name, 'pk': value,
'field': self.remote_field.field_name, 'value': value,
}, # 'pk' is included for backwards compatibility
)

在字段声明中，使用db_constraint=False，例如，

album=ForeignKeyAcrossDb(Singer, db_constraint=False, on_delete=models.DO_NOTHING)